A source code abstract generation method

Abstract

This invention discloses a source code summarization method, specifically including the following steps: First, acquire an open-source dataset and preprocess it to generate an Abstract Syntax Tree (AST) and a SimSBT traversal sequence for each code fragment; next, retrieve other code fragments similar to the target code fragment from a corpus as summarization prototypes; then, refine the summarization prototypes using an extractor, retaining key information and removing redundant content; finally, generate the target summary using an editor. This method effectively identifies and corrects redundant and erroneous information in the summary prototypes, and by combining retrieval and generation strategies, the generated summary more accurately reflects the actual function of the code.

Description

Technical Field

[0001] This invention relates to the field of computer technology, and more specifically to a method for generating source code digests. Background Technology

[0002] In recent years, the increasing complexity of software systems has led to a significant rise in software testing and maintenance costs. Research indicates that the maintenance phase accounts for an average of 59% of the total time cost in the software development lifecycle. Code comprehension, a core component of software development and maintenance, directly impacts the smoothness of the development process and the probability of errors. High-quality code summaries not only significantly improve code comprehension efficiency and reduce errors during development, but also play a crucial role in team collaboration. Clear code summaries reduce communication barriers and accelerate team members' understanding of each other's code intent. However, in reality, code summaries are often missing or become outdated due to program iterations, and manually writing and maintaining these summaries is both tedious and time-consuming. Therefore, automatic code summarization has become an important research topic in software engineering and natural language processing.

[0003] Traditional code summarization methods primarily rely on information retrieval techniques. These methods typically construct a corpus of code-summary pairs from open-source software repositories like GitHub or software question-and-answer websites. Then, based on lexical and syntactic similarity metrics, key information is extracted from the source code. Finally, the most similar code segment to the target code fragment is retrieved, and its comments are reused as the summary. While relatively simple, this method has achieved significant success in code summarization, mainly due to the highly structured nature of code summaries and their frequent use of standard phrases such as "return true if," "get the value of," "outputs the," or "throw an exception if." However, traditional information retrieval-based methods still face potential challenges: the limited similarity between retrieved code and source code can lead to significant deviations between the generated summaries and actual needs, especially in datasets with low code reuse rates, where the generated summaries are more likely to contain inaccurate or even erroneous information. Summary of the Invention

[0004] The purpose of this invention is to address the shortcomings of existing technologies by providing a source code digest generation method.

[0005] In a first aspect, the present invention provides a source code digest generation method, comprising the following steps:

[0006] S1: Train the extractor and editor using a dataset related to code summarization;

[0007] S2: Use the trained extractor and editor to perform summary prediction on the source code and generate corresponding natural language descriptions.

[0008] Preferably, step S1 includes the following steps:

[0009] S11: Obtain open-source datasets, preprocess them, generate abstract syntax tree (AST) sequences for each code snippet, and build a retrieval corpus;

[0010] S12: Based on similarity, retrieve similar codes from the retrieval corpus to obtain the top k highly similar summary prototypes; the similarity includes semantic similarity, lexical similarity, and syntactic similarity;

[0011] S13: Refine the summary prototype obtained from S12, retain keywords and remove irrelevant information, and establish the summary framework;

[0012] S14: Treat the summary framework obtained in S13 as a soft template and generate the target summary under the guidance of the source code.

[0013] Preferably, step S11 includes the following steps;

[0014] S111: Convert the source code in the open-source dataset into an abstract syntax tree structure, traverse the type nodes of the abstract syntax tree to obtain an ordered sequence to enhance the structural information of the source code; match the ordered sequence with the source code to construct a new dataset;

[0015] S112: Divide the new dataset into a training set, a test set, and a validation set, and treat the training set as a retrieval corpus.

[0016] Preferably, step S12 includes the following steps:

[0017] S121: For a given source code, extract the embeddings of the pre-trained model as semantic feature vectors, and calculate the semantic similarity by the L2 distance between the semantic feature vectors;

[0018] S122: Calculate the Jaccard similarity between any two pieces of source code, which is the lexical similarity.

[0019] S123: Traverse the abstract syntax tree to obtain an ordered sequence, calculate the edit distance of the ordered sequence, and use it as the syntax similarity;

[0020] S124: Based on the semantic similarity obtained in S121, select n candidate similar code summary pairs. Then, based on the comprehensive scores of lexical similarity and syntactic similarity obtained in S122 and S123, select k most similar code summary pairs from the n candidate similar code summary pairs as summary prototypes; n > k.

[0021] Preferably, step S13 includes the following steps:

[0022] S131: Extract the common substring from the summary prototype and the target summary, mark the words in the common substring as "1" and the other words as "0", forming the label array that the extractor needs to predict. The extractor training data is generated from the summary prototype, source code, and the corresponding label array.

[0023] S132: Use the extractor training data to train the extractor, realize the relationship between the summary prototype and the source code, and construct the summary framework by predicting whether each word in the summary prototype should be retained.

[0024] Preferably, step S132 includes the following steps:

[0025] S1321: Input both the abstract prototype and the source code into the extractor to obtain the corresponding vector representation;

[0026] S1322: Pass the vector representation obtained in S1321 to the linear classification layer to evaluate the probability that each word in the summary prototype is retained;

[0027] S1323: Generate a summary framework based on the prediction results of S1322.

[0028] Preferably, S14 specifically includes:

[0029] After concatenating the summary framework and source code, input the result into the editor to obtain the target summary;

[0030] Preferably, step S2 includes the following steps:

[0031] S21: Use a search algorithm to find the k closest code snippets in the retrieval corpus and obtain k summary prototypes;

[0032] S22: Refine the found summary prototypes and input them into the extractor to obtain k summary frames;

[0033] S23: Select one summary frame from the k summary frames, input it along with the source code into the editor, and output the predicted summary result.

[0034] Preferably, the pre-trained model used for semantic similarity calculation is the CodeBert model, the pre-trained model used in the extractor is the GraphCodeBert model, and the editor uses the CodeT5 model.

[0035] Secondly, the present invention provides a source code summarization generation system, including a trained extractor and an editor, to predict the summary of source code and generate corresponding natural language descriptions.

[0036] Compared with existing technologies, the source code annotation generation method disclosed in this invention has at least the following beneficial effects:

[0037] 1. By refining the abstract prototype to extract a structured framework and organically combining it with the specific content of the source code, this method serves as the input information for the model. This approach effectively distinguishes and integrates key information in the search results while excluding irrelevant content, thus overcoming the problem of relevant and irrelevant information being mixed in traditional information retrieval methods.

[0038] 2. Combining search technology with deep learning technology not only improves the model's attention to low-frequency words but also enhances its generalization ability, which helps generate higher quality and more comprehensive summaries.

[0039] 3. By training the model using different summarization frameworks, effective data-level enhancements are achieved, significantly improving the model's diversity learning ability and thus strengthening its generalization performance. Attached Figure Description

[0040] Figure 1 This is a flowchart of the method of the present invention.

[0041] Figure 2 This is a flowchart illustrating the training process of the extractor and editor of this invention.

[0042] Figure 3 This is a flowchart illustrating the construction process of the prototype abstract of this invention.

[0043] Figure 4 This is a flowchart illustrating the construction process of the abstract framework of this invention.

[0044] Figure 5 This is a flowchart illustrating the construction process of the objective abstract of this invention.

[0045] Figure 6 This is a test flowchart for the extractor and editor of this invention.

[0046] Figure 7 This is one of the embodiments of the method of the present invention.

[0047] Figure 8 This is one embodiment of the AST structure diagram and SimSBT traversal sequence diagram construction process of the present invention.

[0048] Figure 9 This is one embodiment of the process of constructing the abstract framework diagram of the present invention. Detailed Implementation

[0049] The specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0050] As can be seen from the background technology, traditional information retrieval-based methods still face potential challenges: the similarity between the retrieved code and the source code is limited, which may lead to a large deviation between the generated summary and the actual needs, especially in datasets with low code reuse rates, where the generated summary is more likely to contain inaccurate or even erroneous information.

[0051] Therefore, this invention provides a source code summarization method that aims to solve the information mixing problem in traditional information retrieval methods by integrating the structural information of similar summaries with the specific content of the source code. Specifically, firstly, retrieved similar summaries are used as prototypes to filter out keywords and phrases closely related to the current summarization task. Addressing potential inconsistencies between the prototype and the target summary, irrelevant information in the prototype is removed, and an abstract retrieval framework is extracted. Next, this framework is combined with the source code to provide richer input to the pre-trained model, filling in the specific vocabulary within the abstract framework. By appropriately adjusting the similar summaries, it is possible to generate target summaries that meet specific needs, where the similar summaries provide the basic syntactic framework, while the source code supplements specific lexical information. This process not only improves the quality of the input information but also provides more precise guidance to the model, enabling it to more effectively utilize existing corpora for input augmentation and generate more accurate and coherent target summaries. This invention can effectively identify and correct redundant and erroneous information in the summary prototype, and by combining retrieval and generation strategies, the generated summary more accurately reflects the actual function of the code.

[0052] Specifically, this embodiment relates to a source code digest generation method, see [link to relevant documentation]. Figure 1 The method mainly includes two steps: a training phase (S1) and a generation phase (S2). The training phase uses massive amounts of high-quality data to train the model; the generation phase applies the trained model to predict summaries of the source code and outputs natural language descriptions. See one embodiment of the method of this invention. Figure 7 .

[0053] S1 specifically includes the following steps:

[0054] S11: Obtain a large amount of open-source dataset, preprocess the dataset to obtain the AST for each code snippet, and use the SimSBT method to traverse the dataset and extract the AST sequence representing the code structure. See also Figure 2 The specific steps are as follows:

[0055] S111: Obtain a high-quality source code dataset from the open-source platform GitHub, use JavaParser to convert code snippets into an abstract syntax tree (AST) structure, and use the SimSBT method to traverse type nodes to obtain an ordered sequence to enhance the structural information of the source code; match the ordered sequence with the source code to construct a new dataset; the AST and traversal sequence implementation is as follows: Figure 8 As shown.

[0056] S112: Divide the new dataset into training set, test set, and validation set according to 80%, 10%, and 10% respectively. The training set will be used as the retrieval corpus for subsequent retrieval methods.

[0057] S12: Retrieve similar codes from the corpus based on semantic similarity, lexical similarity, and syntactic similarity to obtain the top-k summary prototypes. See [link to relevant documentation]. Figure 3 The details are as follows:

[0058] S121: Input the source code into the pre-trained model to obtain the embedding matrix of the code, where each row represents a word vector, thus obtaining the semantic feature vector. This paper uses CodeBert as the pre-trained model and combines it with BERT-Whithening operations to further optimize the semantic vector representation. BERT-Whithening helps reduce storage space, improve the model's retrieval speed, and optimize performance. The processed vector representation is saved to build a semantic repository for subsequent retrieval. Semantic similarity is measured by calculating the L2 distance between given codes. The calculation formula is as follows:

[0059]

[0060] in and Let A and B represent the feature vectors of code snippets A and B after optimization using BERT-Whitening, respectively, where d is the dimension preserved by the BERT-Whitening operation.

[0061] S122: Segment the given code snippet according to camelCase naming rules, remove duplicates to form a tag set, and use Jaccard similarity to calculate the similarity between the two sets, which will serve as the metric for lexical similarity. The formula is as follows:

[0062]

[0063] in and These represent the sets of tags for code snippets A and B, respectively.

[0064] S123: Syntactic similarity is evaluated by calculating the edit distance of the SimSBT traversal sequence of the abstract syntax tree, as shown in the following formula:

[0065]

[0066] in Indicates the Levenshtein distance;

[0067] S124: First, based on the semantic similarity calculated in S121, select the top n most similar candidate codes from the corpus. Then, combine this with lexical similarity... and grammatical similarity Calculate the overall score for each candidate code. The higher the score, the better. Select the k most similar code snippets from the candidate code and use their corresponding summaries as the summary prototype.

[0068] Where 𝜆 is a hyperparameter used to adjust the importance of lexical and syntactic similarity. This represents two source code snippets.

[0069] S13: Refine the summary prototype obtained in S12, retaining key information and removing irrelevant parts to construct the summary framework. See also Figure 4 The specific steps are as follows:

[0070] S131: In constructing the model training data, the following strategy was adopted: The model input consists of code snippets and retrieved summary prototypes, and the target output is a label array. Common substrings are extracted from the summary prototype and the target summary, and these words are marked as "1", while other words are considered mismatched and marked as "0", thus forming the label array that the model needs to predict. Finally, a summary framework is generated based on the summary prototype and the label array, that is, the parts labeled "0" are replaced with model placeholders.

[0071] S132: Use an extractor to capture the relationship between the summary prototype and the input code snippet. The code snippet and the summary prototype are merged and fed into the extractor, which predicts whether each word in the prototype should be retained, thus constructing the final summary framework.

[0072] Figure 9 The diagram illustrates an embodiment of the abstract framework construction process. A detailed flowchart of step S132 in this method further includes:

[0073] S1321: After word-by-word decomposition of the abstract prototype sequence and code sequence, they are jointly input into the pre-trained model in the extractor to obtain the corresponding vector representation. The pre-trained model used in this method is GraphCodeBert, and a linear classification layer is added above its output layer to adapt to the classification task. The formula is as follows:

[0074]

[0075] in This indicates the model embedding result. This represents the input code sequence. This represents the prototype of the summary obtained from S124;

[0076] The extractor described above consists of a pre-trained model and a linear classification layer.

[0077] S1322: Represent the vector obtained from S1321 The results are passed to a linear classification layer, which is responsible for evaluating the probability that each word in the summary prototype will be retained. If the retention probability of a word is higher than a preset threshold, the label at the corresponding array position is set to "1"; otherwise, it is set to "0". The formula is as follows:

[0078]

[0079] in and There are two trainable parameters;

[0080] S1323: Generate a summary framework based on the label array predicted in S1322. Specifically, for each word in the label array corresponding to the summary prototype, replace the word at the position marked "0" with a model placeholder.

[0081] The implementation process for constructing the summary framework can be found in [reference needed]. Figure 9 .

[0082] S14: Treat the summary framework obtained in S13 as a soft template and generate the target summary under the guidance of the input code. See also Figure 5 The specific steps are as follows:

[0083] S141: Construct a new dataset based on the summary framework generated in S1323. Concatenate the input code snippet with the selected summary framework into a single input sequence, and separate them using a special marker (such as [SEP]) to ensure that the model can clearly distinguish between the two parts.

[0084] S142: During training, each summarizing frame is combined with its associated input code into a data record. Notably, the same code segment may be associated with multiple frames, generating multiple data records, each of which produces a corresponding summary, thus augmenting the training data. This augmentation strategy enables the model to learn diverse summarizing frames, thereby improving its generalization ability.

[0085] S2: Utilize a trained extractor and editor to predict the source code summary and generate a corresponding natural language description. See also Figure 6 The details are as follows:

[0086] S21: Use a search algorithm to locate the k closest code snippets in the previously constructed retrieval corpus and extract the corresponding summaries as summary prototypes.

[0087] S22: Refine the retrieved summary prototype and input it into the extractor to generate a summary framework.

[0088] S23: Select the best-fitting frame from multiple candidate frames to generate the summary. The selection criterion is mainly based on the composition of the tag array in the frame: a preference is given to frames containing more "1"s (reserved tokens) and fewer "0"s (ignored tokens). Finally, the selected summary frame and the source code are input into the editor, and the predicted summary result is output. This article uses CodeT5 as the editor.

[0089] The source code digest method disclosed in this invention is applicable to software maintenance scenarios. When existing software needs maintenance and updates, source code digests help maintenance personnel quickly familiarize themselves with the functionality and structure of the code, thereby enabling more accurate modifications and optimizations and reducing maintenance costs.

[0090] Specifically, software maintenance methods may include, for example:

[0091] To identify problems or areas for improvement in the software;

[0092] Obtain the source code of the software that needs to be maintained from the version control system;

[0093] A source code digest generation tool is used to generate a digest for the entire software system. This source code digest generation tool can call the source code digest method involved in this invention.

[0094] By reading source code summaries, you can quickly understand the overall architecture of the software and the code logic of the problem-related parts.

[0095] Conduct a detailed review of the code related to the problem, analyze possible causes of the problem and solutions;

[0096] Based on the results of problem analysis and code review, a specific maintenance plan will be developed;

[0097] Define the maintenance objectives, steps, timeline, and required resources;

[0098] The source code was modified and optimized according to the maintenance plan.

[0099] The present invention can also provide an electronic device, specifically, the electronic device includes a memory and a processor, wherein the memory stores executable code, and when the processor executes the executable code, it implements the method described in any of the embodiments.

[0100] The memory may include high-speed random access memory (RAM) or non-volatile memory, such as at least one disk drive. Communication between this system network element and at least one other network element is achieved through at least one communication interface (which can be wired or wireless), such as the Internet, wide area network, local area network, or metropolitan area network.

[0101] The bus can be an ISA bus, PCI bus, or EISA bus, etc. The bus can be divided into address bus, data bus, control bus, etc.

[0102] The memory is used to store programs. After receiving an execution instruction, the processor executes the program. The method executed by the device for defining the flow process disclosed in any of the foregoing embodiments of the present invention can be applied to the processor or implemented by the processor.

[0103] The processor may be an integrated circuit chip with signal processing capabilities. In implementation, the steps of the above methods can be completed through integrated logic circuits in the processor's hardware or through software instructions. The processor can be a general-purpose processor, including a Central Processing Unit (CPU) and a Graphics Processing Unit (GPU). For large-scale model inference, GPUs offer significant advantages as processors because they can accelerate the inference process of neural network models through high parallel computing capabilities. For complex deep learning tasks or inference involving large models, GPUs are ideal; however, in resource-constrained scenarios or with smaller inference scales, CPUs can still be used as processors to execute tasks. The methods, steps, and logic diagrams disclosed in the embodiments of this invention can be implemented or executed. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of this invention can be accelerated by hardware accelerators (such as GPUs, TPUs, FPGAs, etc.) or executed by a combination of hardware and software modules in a decoding processor. The software module can reside in a mature storage medium in the field, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, or registers. This storage medium is located in memory, and the processor reads information from the memory and, in conjunction with its hardware, completes the steps of the above method.

[0104] The embodiments of the present invention may also provide a computer program product of a readable storage medium, including a computer-readable storage medium storing program code. The instructions included in the program code can be used to execute the methods described in the preceding method embodiments. For specific implementation, please refer to the foregoing method embodiments, which will not be repeated here.

[0105] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, essentially, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0106] In summary, the source code digest generation method of the present invention achieves excellent technical results in extracting source code digests, bringing significant improvements to fields such as software development and maintenance.

[0107] This invention integrates semantic, lexical, and syntactic similarity to retrieve similar code and obtain abstract prototypes. Semantically, it uses pre-trained model embedding vectors to calculate L2 distance; lexically, it segments words according to camelCase naming and uses Jaccard similarity measurement; syntactically, it evaluates the distance by traversing the sequence of an abstract syntax tree. This multi-dimensional similarity consideration collaboratively ensures accurate retrieval and lays the foundation for high-quality abstracts.

[0108] This invention analyzes common substrings from the prototype and target abstract, constructs a tag array using "1" and "0" markers to generate an abstract framework, accurately retains key words and removes irrelevant information, making the abstract core prominent, concise and accurate in reflecting the code function, and achieving precise extraction of key information.

[0109] This invention uses the CodeBert model for pre-training to optimize semantic vectors, selects GraphCodeBert as the extractor to capture the relationship between prototypes and code, and uses CodeT5 as the editor to generate summaries. Through CodeBert's strong semantic parsing, GraphCodeBert's excellent relationship mining, and CodeT5's superior text generation, the quality and accuracy of summaries are jointly improved.

[0110] Therefore, traditional information retrieval methods suffer from summary bias due to low similarity between the retrieved code and the source code. This invention refines the prototype into a framework and combines it with the source code to differentiate and integrate information, overcoming the drawback of mixed good and bad search results and ensuring that the summary accurately reflects the actual function of the code. It integrates search and deep learning technologies to improve the processing capability of low-frequency words. It accurately identifies and extracts low-frequency professional terms in specific fields, avoiding the omission of key information, ensuring that the summary fully and accurately conveys the semantics of the code, and enhancing the comprehensiveness and depth of the summary.

[0111] In addition to the embodiments described above, the present invention may have other implementations. All technical solutions formed by equivalent substitution or equivalent transformation are within the scope of protection claimed by the present invention.

[0112] The present invention has been described in detail above, but its specific implementation is not limited thereto. Various modifications or alterations can be made by those skilled in the art without departing from the spirit and scope of the claims of this application.

Claims

1. A method for generating source code digests, characterized in that, Includes the following steps: S1: Train the extractor and editor using a code summarization dataset; the extractor consists of a pre-trained model and a linear classification layer, with the pre-trained model being the GraphCodeBert model; use CodeT5 as the editor; including the following steps: S11: Obtain open-source datasets, preprocess them, generate abstract syntax tree sequences for each code snippet, and build a retrieval corpus; S12: Based on similarity, retrieve similar codes from the retrieval corpus to obtain the top k highly similar summary prototypes; the similarity includes semantic similarity, lexical similarity, and syntactic similarity; S13: Refine the summary prototype obtained in S12, retaining keywords and removing irrelevant information to establish the summary framework; specifically: S131: Extract the common substring from the summary prototype and the target summary, mark the words in the common substring as "1" and the other words as "0", forming the label array that the extractor needs to predict. The extractor training data is generated from the summary prototype, source code, and the corresponding label array. S132: Use the extractor training data to train the extractor, capture the relationship between the summary prototype and the source code, and build the summary framework; S14: Treat the summary framework obtained in S13 as a soft template, and generate the target summary under the guidance of the source code. Specifically, the result of concatenating the summary framework and the source code is input into the editor to obtain the target summary. S2: Use the trained extractor and editor to perform summary prediction on the source code and generate corresponding natural language descriptions.

2. The method according to claim 1, characterized in that, S11 includes the following steps: S111: Convert the source code in the open-source dataset into an abstract syntax tree structure, traverse the type nodes of the abstract syntax tree to obtain an ordered sequence to enhance the structural information of the source code; match the ordered sequence with the source code to construct a new dataset; S112: Divide the new dataset into a training set, a test set, and a validation set, and treat the training set as a retrieval corpus.

3. The method according to claim 2, characterized in that, S12 includes the following steps: S121: For a given source code, extract the embeddings of the pre-trained model as semantic feature vectors, and calculate the semantic similarity by the L2 distance between the semantic feature vectors; S122: Calculate the Jaccard similarity between any two pieces of source code, which is the lexical similarity; S123: Traverse the abstract syntax tree to obtain an ordered sequence, calculate the edit distance of the ordered sequence, and use it as the syntax similarity; S124: Based on the semantic similarity obtained in S121, select n candidate similar code summary pairs. Then, based on the comprehensive scores of lexical similarity and syntactic similarity obtained in S122 and S123, select k most similar code summary pairs from the n candidate similar code summary pairs as summary prototypes; n > k.

4. The method according to claim 3, characterized in that, S132 includes the following steps: S1321: Input the summary prototype and source code into the pre-trained model simultaneously to obtain the corresponding vector representation; S1322: Pass the vector representation obtained in S1321 to the linear classification layer to evaluate the probability that each word in the summary prototype is retained; S1323: Generate a summary framework based on the prediction results of S1322.

5. The method according to claim 1, characterized in that, S2 includes the following steps: S21: Find the k closest code snippets in the retrieval corpus and obtain k summary prototypes; S22: Refine the found summary prototypes and input them into the extractor to obtain k summary frames; S23: Select one summary frame from the k summary frames, input it along with the source code into the editor, and output the predicted summary result.

6. The method according to claim 1 or 3, characterized in that, The pre-trained model used for semantic similarity calculation is the CodeBert model.

7. A source code digest generation system based on the method of any one of claims 1-6, characterized in that... It includes a trained extractor and editor to perform summary prediction of source code and generate corresponding natural language descriptions.

Images