A personalized API recommendation method based on developers' historical code

Abstract

The present invention provides a personalized API recommendation method based on developer historical codes, and relates to the field of network recommendation technology. The present invention collects training samples that not only include projects jointly developed by a large number of developers that conform to the programming habits of general developers, but also collects the developer's personal historical project codes to make personalized API recommendations; uses data enhancement to process training sample vectors, and adds a sample weighting module to the GREAT model to achieve data enhancement of the developer's personal historical code. Through personalized recommendations based on developer historical projects, the model will learn the developer's unique personalized API preference characteristics, and can better consider the developer's personal preferences for API use when writing code. By taking the developer's personal preferences into consideration during the recommendation process, the developer is more likely to accept the recommended API, thereby greatly improving the accuracy of the generated recommendation results.

Description

Technical Field

[0001] The present invention relates to the field of network recommendation technology, and in particular to a personalized API recommendation method based on developer historical codes. Background Art

[0002] With the rapid development of computer technology, the development requirements and difficulty of various software and applications are constantly increasing. With the help of APIs, developers can complete their programming tasks more efficiently. However, due to the large number of APIs in the library, it is difficult for developers to be familiar with all APIs and quickly select the correct API for a specific development task. The research and application of API recommendation technology has great research potential and value. API recommendation technology can quickly provide developers with accurate API methods to deliver feature-rich software, and to a certain extent, it guarantees the practicality of the method, greatly saving developers' time and improving the efficiency of project and software development.

[0003] There are currently many API recommendation methods and plug-ins to reduce the burden on developers in understanding and searching for APIs. Based on different input methods, API recommendation scenarios are generally divided into two categories, namely query-based recommendations and code-based recommendations. The first type of recommendation method requires developers to enter a natural language query about the API. In the second recommendation method, since there is no clear query statement, adjacent code snippets will be used as context, and possible missing APIs will be inferred and recommended to developers. Most related work adopts text similarity-based techniques. For example, some work recommends APIs based on the similarity between search queries and API supplementary information.

[0004] Previous methods rarely pay attention to the role of developers' personalized API usage implicit in their historical code in the recommendation process, which is often crucial to improving API recommendation performance.

[0005] Chinese patent "CN 202010723230.6 API Recommendation Method Based on Deep Learning and Code Context Structure and Text Information" provides a deep learning neural network that indirectly integrates code text information and code structure information. It uses a gated graph neural network (GGNN) to construct an API context graph network to learn code structure information, and a code token network to learn code text information. Finally, a joint layer is used to fuse these two types of information to learn the textual and structural features of the code. GGNN (gated graph neural network) relies on explicit message passing to achieve structural inductive bias and can only indirectly incorporate structural and textual global information into the model. Summary of the Invention

[0006] The technical problem to be solved by the present invention is to address the deficiencies of the above-mentioned existing technologies and provide a personalized API recommendation method based on developer historical code, which can effectively use developer historical project information and does not rely on explicit message passing to achieve structural inductive bias.

[0007] In order to solve the above technical problems, the technical solution adopted by the present invention is:

[0008] A personalized API recommendation method based on developers' historical code. This method redesigns the deep learning neural network model by enhancing developers' personal code data in the training set to learn developers' preferences for library APIs.

[0009] The collected training samples not only include projects developed by a large number of developers that conform to the programming habits of the general public, but also collect the developer's personal historical project code to make personalized API recommendations;

[0010] Use data augmentation to increase the influence of developers' personal code data on deep learning neural network model training. That is, by using sample weighting, the importance of developers' personal historical project sample data in the training model is increased, thereby strengthening the deep learning neural network model's learning of the APIs used by developers and their usage sequences.

[0011] If an API token in a developer's personal historical project data appears less frequently in the crawled corpus, then the API token reflects the developer's unique programming preferences, so the weight of the API token should be higher during training;

[0012] Use formula (1) to calculate the weight of an API token;

[0013] Weight(x)=TotalNum / Count(x) (1)

[0014] Among them, Count(x) is the number of times a certain API token appears in the total corpus. If its value is 0, it will be corrected to 1; TotalNum is the number of times a certain API token appears in the developer's personal corpus;

[0015] In the design of deep learning neural network models, the graph relation embedding attention model and the code pointer transformation model, namely the GREAT model and the TOCOPO model, are combined for API recommendation problems.

[0016] The following steps are involved:

[0017] Step 1: Crawl multi-developer project codes and developer personal historical codes on GitHub;

[0018] Step 2: Preprocess training samples;

[0019] Step 3: Construct a deep learning neural network that directly uses code structure and text information;

[0020] Step 4: Train the deep learning neural network model;

[0021] Step 5: Use the trained deep learning neural network model for code recommendation.

[0022] The beneficial effects of adopting the above technical solution are: the personalized API recommendation method based on the developer's historical code provided by the present invention uses data enhancement to process the training sample vector, and realizes the enhancement of the developer's personal historical code data by adding a sample weighting module to the GREAT model. Combining the GREAT model and the TOCOPO model, it is not necessary to rely on explicit message passing, and the OOV (Out of vocabulary) problem of the token is solved through a copy mechanism. Compared with the existing technology, through personalized recommendations based on the developer's historical projects, the model will learn the developer's unique personalized API preference characteristics, and can better take into account the developer's personal preferences for API use when writing code. Taking the developer's personal preferences into consideration during the recommendation process, the developer is more likely to accept the recommended API, thereby greatly improving the accuracy of the generated recommendation results. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1 A flowchart of the personalized recommendation model training provided by an embodiment of the present invention;

[0024] Figure 2 The code provided by the embodiment of the present invention is converted into an AST path representation graph. DETAILED DESCRIPTION

[0025] The following embodiments of the present invention are described in further detail with reference to the accompanying drawings and examples. The following examples are used to illustrate the present invention but are not intended to limit the scope of the present invention.

[0026] If developers' personal preferences are taken into account during the recommendation process, they are more likely to accept the recommended APIs. Current API recommendation methods fail to account for differences in developers' API preferences. This work aims to provide personalized API recommendations for developers. Therefore, we propose redesigning the deep learning model by augmenting the training set with developers' personal code data to learn their preferences for library APIs.

[0027] This embodiment designs a personalized API recommendation method that uses data enhancement to process training sample vectors. The collected training samples not only include projects jointly developed by a large number of developers that conform to the programming habits of the general developers, but also collect the developer's personal historical project code to make personalized API recommendations. Data enhancement is used to increase the influence of the developer's personal code data on the training of the deep learning model, that is, the importance of the developer's personal historical project sample data in the training model is increased by sample weighting to enhance the deep learning model's learning of the APIs used by the developer and their usage sequences. If an API token in the developer's personal historical project data appears less frequently in the crawled corpus, then the API token will reflect the developer's unique programming preferences, so the weight of the API token should be higher during training. The weight of an API token is calculated using the formula Weight(x) = TotalNum / Count(x), where Count(x) is the number of times an API token appears in the total corpus (if its value is 0, it is corrected to 1), and TotalNum is the number of times it appears in the developer's personal corpus.

[0028] The Graph Relational Embedding Attention Transformers (GREAT model) can introduce the characteristics of the edge type between nodes into the calculation of the attention mechanism module by changing the attention calculation, so that the edge type embedding can be added to each multi-head attention and Head, and can directly learn long-distance code text information and local code structure information. The code pointer transformation model (tocode or pointer output, TOCOPO model) has a built-in copy mechanism that can more easily reuse tokens that appear less frequently in the input. This mechanism can select subsequences in the input sequence and place them in the appropriate position in the output sequence to solve the OOV (Out of vocabulary) problem of tokens. Therefore, this embodiment combines the advantages of the GREAT model and the TOCOPO model for API recommendation problems.

[0029] like Figure 1 As shown, the method of this embodiment is as follows.

[0030] Step 1: Crawl the multi-developer project code and the developer's personal historical code on GitHub; follow the steps below:

[0031] Step 1.1: Obtain Java project codes from various fields developed by a large number of developers from the open source community platform GitHub and create a large corpus from GitHub. The specific method is as follows:

[0032] Step 1.1.1: Use the crawler library to download code resources;

[0033] Use the requests library to download various link resources from the searched URL to a local folder and decompress them;

[0034] Step 1.1.2: Filter out projects that only have documentation or do not include enough code;

[0035] Use the cloc tool to scan the data in each project and filter out projects with less than 10 files or less than 1000 lines of code, or projects with Java code but the proportion is less than 10%;

[0036] Step 1.2: Collect developers’ historical project code data from the open source community platform GitHub to train the personalized API recommendation model;

[0037] We selected outstanding GitHub contributors with more than 5 independently written projects and more than 10,000 followers, and excluded those with non-code usable project data larger than 20Mb and their Java projects. The specific method is:

[0038] Step 1.2.1: Also use the crawler library to download code resources;

[0039] Step 1.2.2: Also filter out projects that only have documentation or do not include enough code;

[0040] Use the cloc tool to scan the data in each project in the personal code data and filter according to the criteria in step 1.1.2.

[0041] Step 2: If Figure 2 As shown, the training samples are preprocessed for training deep learning networks.

[0042] Step 2.1: Use JavaPaser to convert the source code into AST and save it as a c2s file.

[0043] All the codes collected in step 1 are analyzed and extracted into AST form by Java compiler, and stored in the form of c2s files by line. The method name of each method is the first item, and the rest of the AST path is separated by commas. Each AST path has three parts: the head token, the middle node, and the tail token. These three parts are separated by vertical bars.

[0044] Step 2.2: Convert the c2s format AST path data file stored in rows into a graph-structured json file;

[0045] Step 2.2.1: Treat each subtoken in the c2s file as a node, with the first node type as START_AST_NODE, the middle node type as AST_NODE, and the last node type as END_NODE;

[0046] Step 2.2.2: Add edges for every two adjacent nodes in each AST path;

[0047] Add a directed edge of type START_AST_PATH between the first node and the intermediate node; add directed edges of type NEXT_AST_PATH between the intermediate nodes in sequence; add a directed edge of type END_AST_PATH from the tail node to the intermediate node;

[0048] Step 2.2.3: Generate a ROOT_AST_PATH type node for each AST path, and generate a directed edge pointing to the AST path node with all nodes of the corresponding AST path:

[0049] Generate an edge of type START_TOKEN with the first node; generate an edge of type AST_NODE with the middle node; and generate an edge of type END_TOKEN with the last node;

[0050] Step 2.2.4: Generate a total ROOT node with directed edges of type AST_PATH pointing to all ROOT_AST_PATH nodes;

[0051] Step 2.2.5: Finally, a graph model json format file consisting of AST path is generated;

[0052] Step 2.3: Generate a dictionary and its word frequency table for each node and edge type:

[0053] Generate dictionaries and word frequencies for edge types, node labels, node types, and output tokens, and rank them in descending order of frequency.

[0054] Step 2.4: Pre-encode the string-type token in the JSON format file generated in step 2.2.5 into a number:

[0055] The generated edge type, node label, node type, and output token are replaced according to the word frequency number in the dictionary file for preliminary preprocessing encoding;

[0056] Step 2.5: Serialize the file encoded in step 2.4 into a TFRecord format file:

[0057] Use TensorFlow's TFRecordWriter function to serialize the file encoded in step 2.4 into a TFRecord format file.

[0058] Step 3: Construct a deep learning neural network that directly uses code structure and text information, including the following steps:

[0059] Step 3.1: Build the GREAT model encoder for API recommendations by:

[0060] Step 3.1.1: The embedding layer of the GREAT model converts the API represented by each node in the AST path graph into a 128-dimensional vector;

[0061] Step 3.1.2: Sample weighting. Change the weight of the code API token in the developer's historical project. The formula for calculating a certain API token is as follows:

[0062] Weight(x)=TotalNum / Count(x) (1)

[0063] Where Count(x) is the number of times a certain API token appears in the multi-developer project corpus (if its value is 0, it will be changed to 1), and TotalNum is the number of times a certain API token appears in the developer's personal historical project code;

[0064] Step 3.1.3: Learn the global information of all vectors through the multi-head attention mechanism. The attention calculation formula is:

[0065]

[0066] Among them, q i and k j are the query and key vectors of API token i and API token j, respectively, so the relation features can be directly incorporated into the attention function; e ij is the attention score between API token i and API token j; b ij is an additional bias term for the specific attention weight between API token i and API token j; N is the attention dimension of each head, and T represents the transpose. ij The calculation formula is:

[0067]

[0068] Where e is the word embedding vector of the edge type connecting nodes i and j, e∈RN ;W e is the weight matrix of the e vector, W e ∈R N ; b e is the deviation vector of the e vector, b e ∈R;

[0069] If there are multiple edges between two nodes, then multiple b ij Perform simple addition to obtain the final deviation term;

[0070] Step 3.1.4: Output the output vector of the multi-head attention mechanism module to the feedforward network layer, perform layer normalization, and then input it into the TOCOPO model.

[0071] Step 3.2: Build the TOCOPO model decoder for API recommendation. The specific method is:

[0072] Step 3.2.1: The TOCOPO model converts each input code token into a 128-dimensional vector through the embedding layer;

[0073] Step 3.2.2: Input the vector obtained in step 3.2.1 into the Masked Multi-Head Attention mechanism to obtain a 128-dimensional vector;

[0074] Step 3.2.3: The output vector of the masked multi-head attention mechanism module and the output vector of the GREAT model are input into the multi-head attention mechanism module (Multi-Head Attention) to obtain a 128-dimensional output vector;

[0075] Step 3.2.4: The 128-dimensional output vector obtained in step 3.2.3 is then input into the feedforward neural network in the TOCOPO model to finally obtain the candidate API token score.

[0076] Step 4: Train the deep learning neural network model. The specific method is as follows:

[0077] Step 4.1: Split all the data sets obtained in steps 1 and 2 into training and validation sets in a ratio of 8:2, and input them into the deep learning neural network designed in step 3 respectively;

[0078] Step 4.2: Input all multi-developer project codes and developers' personal historical codes into the deep learning neural network designed in step 3, perform model parameter training, and obtain a trained deep learning neural network model;

[0079] Step 4.3: If the model fails to improve on the validation set after 10 rounds of continuous training, stop training and take the last model as the final model.

[0080] Step 5: Use the trained deep learning neural network model for code recommendation:

[0081] Step 5.1: Create a plug-in in the code editor according to the methods in steps 1 to 4;

[0082] Step 5.2: The developer opens all the codes and plugins in the code editor;

[0083] Step 5.3: The plugin reads the current position of the developer's cursor in the code editor;

[0084] Step 5.4: Based on the developer's previous project code, the trained deep network model is used to calculate a candidate recommendation list. After the developer clicks the recommendation icon of the plug-in in the code editor, the plug-in will display the M recommended APIs calculated by the model to the right of the cursor position; for example, the plug-in will display the top 10 recommended APIs calculated by the model to the right of the cursor position.

[0085] Step 5.5: The developer selects the required API, and the selected API will be filled in behind the developer's cursor position.

[0086] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the above embodiments, or make equivalent replacements for some or all of the technical features therein. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope defined by the claims of the present invention.

Claims

1. A personalized API recommendation method based on developer historical code, characterized by: The method redesigns the deep learning neural network model by enhancing the developer's personal code data in the training set to learn the developer's preference for library API; The collected training samples include projects developed by a large number of developers that conform to the programming habits of general developers, as well as the code of developers' personal historical projects, which are used for personalized API recommendations; Use data augmentation to increase the influence of developers' personal code data on deep learning neural network model training. That is, by using sample weighting, the importance of developers' personal historical project sample data in the training model is increased, thereby strengthening the deep learning neural network model's learning of the APIs used by developers and their usage sequences. If an API token in a developer's personal historical project data appears less frequently in the crawled corpus, then the API token reflects the developer's unique programming preferences, so the weight of the API token should be higher during training; Use formula (1) to calculate the weight of an API token; Weight(x)=TotalNum / Count(x) (1) Among them, Count(x) is the number of times a certain API token appears in the total corpus. If its value is 0, it will be corrected to 1; TotalNum is the number of times a certain API token appears in the developer's personal corpus; In the design of deep learning neural network models, we combine the graph relation embedding attention model and the code pointer transformation model, namely the GREAT model and the TOCOPO model, for API recommendation problems. The method is carried out according to the following steps: Step 1: Crawl multi-developer project codes and developer personal historical codes on GitHub; Step 2: Preprocess training samples; Step 3: Construct a deep learning neural network that directly uses the code structure and text information; specifically, the following steps: Step 3.1: Build the GREAT model encoder for API recommendations by: Step 3.1.1: The embedding layer of the GREAT model converts the API represented by each node in the AST path graph into a 128-dimensional vector; Step 3.1.2: Sample weighting, change the weight of the code API token in the developer's historical project, and calculate the weight of a certain API token according to formula (1); Step 3.1.3: Learn the global information of all vectors through the multi-head attention mechanism; Step 3.1.4: Output the output vector of the multi-head attention mechanism module to the feedforward network layer, perform layer normalization, and then input it into the TOCOPO model; Step 3.2: Build the TOCOPO model decoder for API recommendation. The specific method is: Step 3.2.1: The TOCOPO model converts each input code token into a 128-dimensional vector through the embedding layer; Step 3.2.2: Input the vector obtained in step 3.2.1 into the masked multi-head attention mechanism to obtain a 128-dimensional vector; Step 3.2.3: The output vector of the masked multi-head attention mechanism module and the output vector of the GREAT model are input into the multi-head attention mechanism module to obtain a 128-dimensional output vector; Step 3.2.4: The 128-dimensional output vector obtained in step 3.2.3 is then input into the feedforward neural network of the TOCOPO model to obtain the candidate API token score. Step 4: Train the deep learning neural network model; Step 5: Use the trained deep learning neural network model for code recommendation.

2. The personalized API recommendation method based on developer historical code according to claim 1, characterized in that: The step 1 is specifically performed as follows: Step 1.1: Obtain Java project codes from various fields developed by a large number of developers from the open source community platform GitHub and create a large corpus from GitHub. The specific method is as follows: Step 1.1.1: Use the crawler library to download code resources; Use the requests library to download various link resources from the searched URL to a local folder and decompress them; Step 1.1.2: Filter out projects that only have documentation or do not include enough code; Use the cloc tool to scan the data in each project and filter out projects with less than 10 files or less than 1000 lines of code, or projects with Java code but the proportion is less than 10%; Step 1.2: Collect developers’ historical project code data from the open source community platform GitHub to train the personalized API recommendation model; We selected outstanding GitHub contributors with more than 5 independently written projects and more than 10,000 followers, and excluded those with non-code usable project data larger than 20Mb and their Java projects. The specific method is: Step 1.2.1: Use the crawler library to download code resources; Step 1.2.2: Filter out projects that only have documentation or do not include enough code; Use the cloc tool to scan the data in each project in the personal code data and filter according to the criteria in step 1.1.

2.

3. The personalized API recommendation method based on developer historical code according to claim 2, characterized in that: The specific method of step 2 is: Step 2.1: Use JavaPaser to convert the source code into AST and save it as a c2s file; All the codes collected in step 1 are analyzed and extracted into AST form by Java compiler, and stored in the form of c2s files by line. The method name of each method is the first item, and the rest of the AST path is separated by commas. Each AST path has three parts: the head token, the middle node, and the tail token. These three parts are separated by vertical bars. Step 2.2: Convert the c2s format AST path data file stored in rows into a graph-structured json file; Step 2.2.1: Treat each subtoken in the c2s file as a node, with the first node type as START_AST_NODE, the middle node type as AST_NODE, and the last node type as END_NODE; Step 2.2.2: Add edges for every two adjacent nodes in each AST path; Add a directed edge of type START_AST_PATH between the first node and the intermediate node; add directed edges of type NEXT_AST_PATH between the intermediate nodes in sequence; add a directed edge of type END_AST_PATH from the tail node to the intermediate node; Step 2.2.3: Generate a ROOT_AST_PATH type node for each AST path, and generate a directed edge pointing to the AST path node with all nodes of the corresponding AST path: Generate an edge of type START_TOKEN with the first node; generate an edge of type AST_NODE with the middle node; and generate an edge of type END_TOKEN with the last node; Step 2.2.4: Generate a total ROOT node with directed edges of type AST_PATH pointing to all ROOT_AST_PATH nodes; Step 2.2.5: Finally, a graph model json format file consisting of AST path is generated; Step 2.3: Generate a dictionary and its word frequency table for each node and edge type: Generate dictionaries and word frequencies for edge types, node labels, node types, and output tokens, and rank them in descending order of frequency. Step 2.4: Pre-encode the string-type token in the JSON format file generated in step 2.2.5 into a number: The generated edge type, node label, node type, and output token are replaced according to the word frequency number in the dictionary file for preliminary preprocessing encoding; Step 2.5: Use TensorFlow's TFRecordWriter function to serialize the file encoded in step 2.4 into a TFRecord format file.

4. The personalized API recommendation method based on developer historical code according to claim 3, characterized in that: The attention calculation formula in step 3.1.3 is: Among them, q i and k j are the query and key vectors of API token i and API token j respectively; e ij is the attention score between API token i and API token j; N is the attention dimension of each head; T represents transposition; b ij is the additional bias term for the specific attention weight between API token i and API token j, b ij The calculation formula is: Where e is the word embedding vector of the edge type connecting nodes i and j, e∈R N ;W e is the weight matrix of the e vector, W e ∈R N ; b e is the deviation vector of the e vector, b e ∈R; If there are multiple edges between two nodes, then multiple b ij A simple addition is performed to obtain the final deviation term.

5. The personalized API recommendation method based on developer historical code according to claim 4, characterized in that: The specific method of step 4 is: Step 4.1: Split all the data sets obtained in steps 1 and 2 into training and validation sets in a ratio of 8:2, and input them into the deep learning neural network designed in step 3 respectively; Step 4.2: Input all multi-developer project codes and developers' personal historical codes into the deep learning neural network designed in step 3, perform model parameter training, and obtain a trained deep learning neural network model; Step 4.3: If the model fails to improve on the validation set after 10 rounds of continuous training, stop training and take the last model as the final model.

6. The personalized API recommendation method based on developer historical code according to claim 5, characterized in that: The specific method of step 5 is: Step 5.1: Create a plug-in in the code editor according to the methods in steps 1 to 4; Step 5.2: The developer opens all the codes and plugins in the code editor; Step 5.3: The plugin reads the current position of the developer's cursor in the code editor; Step 5.4: Based on the developer's previous project code, the trained deep network model is used to calculate a candidate recommendation list. After the developer clicks the recommendation icon of the plug-in in the code editor, the plug-in will display M recommended APIs calculated by the model to the right of the cursor position; Step 5.5: The developer selects the required API, and the selected API will be filled in behind the developer's cursor position.

Images