Pull Request priority ranking method based on graph neural network
The complex correlation data structure of PR is constructed through graph neural network, which solves the review difficulty and accumulation problems caused by the growth of PR number, and realizes efficient PR sorting and priority recognition, improving code review efficiency and project stability.
Patent Information
- Application Number
- CN202411059016.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-02
- Publication Date
- 2024-11-19
AI Technical Summary
In multi-person collaborative development, the rapid growth of the number of Pull Requests (PRs) has led to increased review difficulty, and it is difficult for existing technologies to effectively sort and prioritize high-priority PRs, resulting in low code review efficiency and PR accumulation problems.
Graph neural networks (GNNs) are used to sort PR priority. By constructing complex correlation data structures between PRs, integrating node features and neighboring node features, and updating PR node embedding vectors using iterative message delivery mechanism to identify high-priority PRs . The specific steps include crawling PR data, processing data, establishing a baseline model, building a graph structure, training a graph neural network model and evaluating the sorting effect.
Achieve more efficient PR sorting, improve code review efficiency, reduce wait time, reduce the risk of open source projects, overcome the limitations of traditional methods when dealing with complex relationships and large-scale data, and provide better generalization capabilities. and interpretability.
Smart Images

Figure CN118981332A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of software development, and in particular to a Pull Request priority sorting method based on a graph neural network. Background Art
[0002] Pull Request (PR) is a common code review method. As the scale of projects in open source code repositories continues to expand, the number of PRs is gradually increasing, so for reviewers, the difficulty of PR review is increasing.
[0003] In multi-person collaborative development, the review and merging of PRs is very important. Therefore, the sorting of PR priorities is of great significance, which can achieve the purpose of improving the efficiency of code review and solving the problem of PR accumulation. In open source projects, many developers will contribute code at the same time, resulting in a large number of PRs, which require a lot of resources to process. Therefore, in order to improve the efficiency of code review, PRs need to be prioritized so that managers can deal with the most urgent parts first during the review. By prioritizing PRs, some high-priority PRs can be given priority, making the review process more efficient, reducing the waiting time of open source projects, and improving the quality and stability of projects. On the other hand, in some large open source projects, PRs may accumulate due to too many users and an imperfect merging mechanism. At this time, prioritizing PRs can help managers process PRs more quickly and alleviate the trouble of PR accumulation, thereby having a positive impact on the development of open source code hosting platforms.
[0004] The reason why graph neural networks (GNNs) are suitable for Pull Request (PR) prioritization is that they are particularly good at processing data structures with complex associations, and PRs and their related code bases, developer interactions, and project dependencies are just such complex networks. Graph neural networks can capture complex relationships, where nodes can represent PRs, developers, files, or project modules, and edges can represent various relationships, such as dependencies, interaction histories, or related topics. This structure can naturally reflect the relative importance and urgency between PRs. At the same time, node feature integration can be performed. Graph neural networks can integrate the characteristics of the nodes themselves (such as the size of the PR, the number of modified lines of code, and the key modules involved) as well as the characteristics of their neighboring nodes (such as the activity of associated developers and the quality of previous PRs). This information aggregation mechanism helps to evaluate the contextual importance of each PR. In addition, through an iterative message passing mechanism, GNN can gradually update the embedding vector of each PR node, which reflects the information propagation in the entire graph structure. This helps to identify which PRs are high priority because of their location or association factors. Combining these advantages, the present invention uses graph neural networks to sort PR priorities. Summary of the invention
[0005] The purpose of this invention is to propose a PR priority sorting method based on graph neural network and obtain the best sorting effect through ablation experiments.
[0006] To achieve the above object, the technical solution of the present invention is to provide a PullRequest priority sorting method based on graph neural network, such as Figure 1 As shown, the following steps are included:
[0007] S1: Crawl PR data. Crawl PR data of 15 large projects from Github, including PR creation time, closing time, length of PR modified content, etc.
[0008] S2: Process PR data, including processing null and infinite values, data standardization, automatic assignment of default values, and calculation of label values.
[0009] S3: Establish a baseline model and use the baseline model, such as a random forest model, to calculate the corresponding ranking results;
[0010] S4: Establish edge structure. Establish four edge types: "author-author", "reviewer-reviewer", "author-reviewer", and "file-file" to construct the basic structure of the graph;
[0011] S5: Train the graph neural network model. Based on the created edge structure, train the weight parameters of the graph neural network model, and finally use the graph neural network to calculate the ranking result.
[0012] S6: Use multiple evaluation indicators such as NDCG, MRR, and MAP to evaluate the ranking results, and finally compare the ranking effects of graph neural networks and various baselines.
[0013] Preferably, the main process of crawling project PR data in S1 includes:
[0014] S1-1: Project selection, choose the right project and ensure that the amount of PR data is sufficient. There are many projects on the Github platform. When selecting a project, you need to pay attention to some special situations and avoid selecting projects whose repositories have been split, whose repositories are read-only, and whose code reviews have been closed.
[0015] S1-2: Use GitHub API to build a request: GitHub API programmatically accesses repository data, including PRs. The base URL is https: / / api.github.com / repos / {owner} / {repo} / pulls, where {owner} and {repo} are replaced with the username and repository name of the target repository, respectively. Initiate a request: Use a programming language and library (Python's requests library) to send a GET request to the URL built above, and include an access token in the request header for authentication.
[0016] S1-3: Data parsing and storage, Parsing response: Extract the required information based on the returned JSON data, including PR number, creator, title, status, number of comments, merge status, Exception handling: Add exception handling logic to deal with request failure, timeout or API restrictions, Data cleaning: Clean the extracted data and handle issues such as format inconsistency
[0017] Preferably, the main process of project PR data processing in S2 includes:
[0018] S2-1: Abnormal data processing,Abnormal data processing is a key preprocessing step to ensure the quality and reliability of data input into the graph neural network.,Specifically, we first identify and process the null values in the data, and adopt advanced filling strategies to maintain the integrity and meaning of the data.,In addition, for the processing of infinite values, we choose a suitable filling strategy,using the average value for replacement,
[0019] S2-2: Data standardization. In order to improve the efficiency of model training and ensure the comparability between different features, we standardize the processed data. The main purpose of standardization is to scale the feature values. We choose Min-Max Scaling to map all feature values to the [0,1] interval to make it suitable for the standardization method of graph neural networks to optimize model performance.
[0020] S2-3: Calculate label features. The calculation of label features involves calculating data such as whether the PR is merged, the PR life cycle, and the number of PR review rounds as evaluation indicators. The present invention further divides the granularity.
[0021] Whether the PR is merged represents the value of the PR. If the PR is merged, it means that the PR has contributed to the project library, otherwise it has no contribution. If the PR is merged, the corresponding priority score is 4. If it is not merged, the priority score is 0.
[0022] The PR life cycle is the time span from the creation to the closing of the PR. The shorter the time span, the lower the review cost of the PR, which is more conducive to the reviewer to prioritize the review. The present invention sorts each PR in ascending order according to its life cycle, and divides it into five priority scores of 0, 1, 2, 3, and 4 according to the time length interval.
[0023] The number of PR review rounds is the total number of reviews conducted on the PR, representing the complexity and controversy of the PR. The more rounds, the higher the review cost. The present invention sorts each PR in ascending order according to the number of review rounds, and divides it into five priority scores of 0, 1, 2, 3, and 4 according to the range of the rounds in which it is located.
[0024] Preferably, the main process of selecting and calculating the baseline model in S3 includes:
[0025] S3-1: Clarify task requirements and goals: First, clearly define the research questions and goals. The ranking here can be quantified as priority scores or levels, so common models for classification tasks or regression tasks can be used.
[0026] S3-2: Literature review: Review the literature in related fields to understand which baseline models are commonly used in similar tasks, and pay attention to those models that are widely cited, proven to be effective, and can provide benchmark performance levels.
[0027] S3-3: Generality and simplicity of the model: Select some general and simple models as baselines, such as RF, FIFO, etc., because they can quickly provide a basic performance standard and are easy to understand and implement.
[0028] Finally, the present invention selects the baseline models commonly used in previous studies: RankBoost, AdaRank, CA, ListNet, RF, FIFO, SSF, and MCF.
[0029] RankBoost is a ranking learning algorithm based on the Boosting framework. In each round of iteration, RankBoost adjusts the weight of samples according to the results of the previous round, so that misclassified or misranked samples receive more attention in the next round. The final ranking result is the weighted sum of the outputs of all weak sorters.
[0030] AdaRank is also a ranking algorithm based on Boosting, similar to RankBoost, but its core lies in how to more effectively allocate sample weights and select weak learners. It iteratively trains weak classifiers and adjusts sample weights to maximize the target evaluation index.
[0031] Coordinate Ascent (CA): Gradient Ascent Method. This method improves sorting performance by iteratively optimizing model parameters. Only one parameter is updated at each step, and the optimization is performed along the gradient direction of the parameter until convergence.
[0032] ListNet adopts a probabilistic framework to solve the ranking problem, treating the ranking task as predicting the probability distribution of each document being clicked by the user relative to other documents. ListNet optimizes the model by minimizing the cross entropy between the predicted probability distribution and the actual click distribution.
[0033] Random Forests (RF) In LTR, random forests are an integrated method that ranks by building multiple decision trees and combining their predictions. Each tree is trained based on randomly selected features and data subsets, and finally the outputs of all trees are aggregated (e.g., averaged) to form the final ranking result.
[0034] FIFO (First-In-First-Out) means "first in, first out". This means that PRs will be processed in the order in which they were created or submitted.
[0035] SSF (Small-Size-First): This is a hypothetical rule that means "process the PR with the shortest code length first." Under this strategy, project maintainers or reviewers will prioritize reviewing PRs that involve fewer code changes.
[0036] MCF (Most-Commented-First): stands for "Most-Commented-First". This processing mode means that PRs that have caused more discussions or comments will be reviewed first.
[0037] Preferably, the main process of edge building of the graph neural network in S4 includes:
[0038] To build a graph neural network model for PR priority sorting, it is necessary to carefully design the structure and features of the graph, as well as how to capture information through the structure of the graph to predict the priority of PR. Finally, the present invention constructs the following four types of edges:
[0039] Author-Author Edges: If two authors have submitted a PR together, an edge is established between them. The weight of the edge can reflect the frequency of cooperation or the success rate of cooperation.
[0040] Reviewer-Reviewer Edges: Similarly, if reviewers often review PRs together, an edge is established, and the weight can reflect the closeness of cooperation or the degree of consensus.
[0041] Author-Reviewer Edges: When PRs from a specific author are frequently reviewed by a specific reviewer, an edge is created, and the weight can be the historical success rate or speed of their collaboration.
[0042] “File-File” Edges: Edges are established between PRs that have modified the same file. The weight can be the degree of association between files or the similarity of modifications.
[0043] Preferably, the main process of training the graph neural network model in S4 includes:
[0044] Based on the previously calculated node feature data and the data of the edges of the nodes, data is input into the model.
[0045] Perform calculations on a K-layer network, where the input of each layer depends on the output of the previous layer.
[0046] In the network calculation process of each layer, each node v is traversed, the state information of all neighboring nodes of the node is integrated, the importance weight a of node v is calculated, and the context information of node v is calculated by combining the state vector and weight parameter. After a GRU structure, the output of the state vector of node v at the kth layer is obtained.
[0047] For the output of the last layer, a linear output layer is added to calculate the final priority score.
[0048] According to the List-wise strategy, based on the difference between the predicted score and the true score, the model parameters are adjusted and the next epoch training is carried out.
[0049] Finally, it is a comprehensive and standard approach to use multiple evaluation indicators such as NDCG (Normalized Discounted Cumulative Gain), MRR (Mean Reciprocal Rank), MAP (Mean Average Precision) to evaluate the ranking effect of graph neural networks and other baseline models.
[0050] It is a comprehensive and standard practice to use multiple evaluation indicators such as NDCG (Normalized Discounted Cumulative Gain), MRR (Mean Reciprocal Rank), MAP (Mean Average Precision) to evaluate the ranking effect of graph neural networks and other baseline models. The following is a brief description of these indicators and how to apply them for comparison:
[0051] NDCG (Normalized Discounted Cumulative Gain) takes into account the relevance score at each position in the sorted results and gives higher weight to the top-ranked results. It penalizes the correct items ranked lower by calculating the cumulative gain and applying a discount factor (usually log(position+1)), and then normalizes the results so that the perfectly sorted list scores 1. NDCG is suitable for evaluating the accuracy of Top-K recommendations, especially for evaluating results with differences in rank or importance.
[0052] MRR (Mean Reciprocal Rank) focuses on the position where the correct or relevant items are first found, and is calculated as the average of the reciprocal of the first hit positions of all queries.
[0053] MAP (Mean Average Precision) measures the average precision, that is, for each query, it calculates the mean precision of all relevant items when they are recalled in the predicted order, and then takes the average of all queries. It takes into account both precision and recall, and is a good indicator for evaluating the overall performance in ranking tasks.
[0054] A terminal includes a processor, a memory, and a program of a Pull Request priority sorting method based on a graph neural network stored in the memory. When the program of the Pull Request priority sorting method based on a graph neural network is executed by the processor, the steps of the Pull Request priority sorting method based on a graph neural network are implemented.
[0055] A computer-readable storage medium stores computer instructions, which, when executed by a processor, implement the steps of the Pull Request priority sorting method based on a graph neural network.
[0056] Compared with the prior art, the present invention overcomes the limitations of traditional methods by creating edge structures and training graph neural network models, achieving a more superior PR ranking effect and providing a good auxiliary role for reviewers to select PRs with higher priorities. The advantages of the present invention are as follows.
[0057] (1) Graph neural networks can process graph structured data, and the relationship between PRs can be represented as a graph. Graph neural networks can use the graph structure and the connection relationship between nodes to learn the representation of nodes, thereby better capturing the dependency and contextual information between PRs. Traditional machine learning methods usually require feature engineering of the graph structure or convert it into a vector representation, and cannot directly capture the complex relationship between PRs.
[0058] (2) Graph neural networks have advantages in processing large-scale graph data. Graph neural networks can process large-scale graph data through methods such as sampling and graph partitioning, and perform efficient training and reasoning. Traditional machine learning methods need to face the challenges of dimensionality curse and computational complexity.
[0059] (3) As shown in Tables 1, 2, and 3 below, the performance of the graph neural network on 15 models based on the three evaluation indicators NDCG, MRR, and MAP is better than that of other machine learning baseline models, so the graph neural network has the best final ranking effect.
[0060] (4) Graph neural networks can provide a certain degree of interpretability, which can serve as a basis for understanding the model in ranking decisions. By analyzing the importance of nodes and edge weights in graph neural networks, researchers can understand how the model considers the relationship and contextual information between PRs, thereby increasing trust and understanding of the model.
[0061] (5) Graph neural networks have good generalization capabilities when processing unseen PRs. Graph neural networks can learn the graph structure and the relationship between nodes, calculate scores based on new PRs, and thus better infer their priorities. This enables the model to perform well when faced with new PR data, and is suitable for scenarios where multiple PRs are added every day. BRIEF DESCRIPTION OF THE DRAWINGS
[0062] The accompanying drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation of the present invention. In the accompanying drawings:
[0063] Figure 1 is a flow chart of the present invention;
[0064] Figure 2 is a flow chart of step S1 of the present invention;
[0065] Figure 3 is a flow chart of step S2 of the present invention;
[0066] Figure 4 is a flow chart of step S3 of the present invention;
[0067] Figure 5 is a flow chart of step S4 of the present invention;
[0068] Figure 6 It is a flow chart of step S5 of the present invention. DETAILED DESCRIPTION
[0069] The present invention will be further described in detail below in conjunction with the embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention, rather than to limit the present invention.
[0070] Example: See Figure 1-Figure 6 , a Pull Request priority sorting method based on graph neural network, the method comprising the following steps:
[0071] S1: Crawl PR data. Crawl PR data of 15 large projects from Github, including PR creation time, closing time, length of PR modified content, etc.
[0072] S2: Process PR data, including processing null and infinite values, data standardization, automatic assignment of default values, and calculation of label values.
[0073] S3: Establish a baseline model and use the baseline model, such as a random forest model, to calculate the corresponding ranking results;
[0074] S4: Establish edge structure. Establish four edge types: "author-author", "reviewer-reviewer", "author-reviewer", and "file-file" to construct the basic structure of the graph;
[0075] S5: Train the graph neural network model. Based on the created edge structure, train the weight parameters of the graph neural network model, and finally use the graph neural network to calculate the ranking result.
[0076] S6: Use multiple evaluation indicators such as NDCG, MRR, and MAP to evaluate the ranking results, and finally compare the ranking effects of graph neural networks and various baselines.
[0077] Preferably, the main process of crawling project PR data in S1 is as follows: Figure 2 As shown, including:
[0078] S1-1: Project selection, choose the right project and ensure that the PR data volume is sufficient. There are many projects on the Github platform. When selecting a project, you need to pay attention to some special situations, such as avoiding projects where the repository has been split, the repository is read-only, and the code review has been closed.
[0079] S1-2: Use the GitHub API to build a request: The GitHub API programmatically accesses repository data, including PRs. The base URL is https: / / api.github.com / repos / {owner} / {repo} / pulls, where {owner} and {repo} are replaced with the username and repository name of the target repository, respectively. Make a request: Using a programming language and library (Python's requests library), send a GET request to the URL built above, including the access token in the request header for authentication.
[0080] S1-3: Data parsing and storage. Parsing response: Extract the required information based on the returned JSON data, such as PR number, creator, title, status, number of comments, merge status, etc. Exception handling: Add exception handling logic to deal with situations such as request failure, timeout, or API restrictions. Data cleaning: Clean the extracted data and handle issues such as format inconsistency.
[0081] Preferably, the main process of the project PR data processing in S2 is as follows: Figure 3 As shown, including:
[0082] S2-1: Abnormal data processing. In this method, abnormal data processing is a key preprocessing step to ensure the quality and reliability of data input into the graph neural network. Specifically, we first identify and process the null values in the data, and use advanced filling strategies to maintain the integrity and meaning of the data. In addition, for the processing of infinite values, we choose a suitable filling strategy and use the average value for replacement.
[0083] S2-2: Data standardization. In order to improve the efficiency of model training and ensure comparability between different features, we standardize the processed data. The main purpose of standardization is to scale the feature values. We choose Min-Max Scaling to map all feature values to the [0,1] interval, making it suitable for the standardization method of graph neural networks to optimize model performance.
[0084] S2-3: Calculate label features. The calculation of label features involves calculating data such as whether the PR is merged, the PR life cycle, and the number of PR review rounds as evaluation indicators. The present invention further divides the granularity.
[0085] Preferably, the main process of baseline model selection and calculation in S3 is as follows: Figure 4 As shown, including:
[0086] S3-1: Clarify task requirements and goals: First, clearly define research questions and goals. The ranking here can be quantified as priority scores or levels, so common models for classification tasks or regression tasks can be used.
[0087] S3-2: Literature review: Review the literature in related fields to understand which baseline models are commonly used in similar tasks. Pay attention to models that are widely cited, proven to be effective, and can provide a baseline performance level.
[0088] S3-3: Examine the universality and simplicity of the model: Select some models that are highly universal and easy to implement as baselines, such as RF, FIFO, etc., because they can quickly provide a basic performance standard and are also easy to understand and implement.
[0089] Preferably, the main process of building edges in the graph neural network in S4 is as follows: Figure 5 As shown, including:
[0090] To build a graph neural network model for PR priority sorting, it is necessary to carefully design the structure and features of the graph, as well as how to capture information through the structure of the graph to predict the priority of PR. Finally, the present invention constructs the following four types of edges:
[0091] S4-1: “Author-Author Edges”: If two authors have submitted a PR together, an edge is established between them. The weight of the edge can reflect the frequency of cooperation or the success rate of cooperation.
[0092] S4-2: “Reviewer-Reviewer Edges”: Similarly, if reviewers often review PRs together, an edge is established, and the weight can reflect the closeness of cooperation or the degree of consensus.
[0093] S4-3: “Author-Reviewer Edges”: When PRs from a specific author are frequently reviewed by a specific reviewer, an edge is established, and the weight can be the historical success rate or speed of their collaboration.
[0094] S4-4: “File-File Edges”: Edges are established between PRs that have modified the same file. The weight can be the degree of association between files or the similarity of modifications.
[0095] Preferably, the main process of training the graph neural network model in S5 is as follows: Figure 6 As shown, including:
[0096] The graph neural network model used is AttentiveFP, which extends the attention mechanism to graph structure data. The core idea is to obtain the context vector of the target node by paying attention to the neighbors and local environment of the target node. This model can predict the properties of unknown nodes. The main advantage of the AttentiveFP model is that it can adaptively assign different weights to different edges and nodes when learning the representation of the entire graph, thereby more accurately predicting the properties of the nodes. The steps are as follows:
[0097] S5-1: Based on the previously calculated node feature data and node edge data, input data into the model.
[0098] S5-2: Perform calculations on K-layer networks. The input of each layer depends on the output of the previous layer.
[0099] S5-3: In the network calculation process of each layer, traverse each node v, integrate the state information of all neighboring nodes of the node, calculate the importance weight a of node v, and combine the state vector and weight parameter to calculate the context information of node v, and pass through a GRU structure to obtain the output of the state vector of node v at the kth layer.
[0100] S5-4: For the output of the last layer, add a linear output layer to calculate the final priority score.
[0101] S5-5: According to the List-wise strategy, based on the difference between the predicted score and the true score, the model parameters are adjusted and the next epoch training is carried out.
[0102] Finally, it is a comprehensive and standard approach to use multiple evaluation indicators such as NDCG (Normalized Discounted Cumulative Gain), MRR (Mean Reciprocal Rank), MAP (Mean Average Precision) to evaluate the ranking effect of graph neural networks and other baseline models.
[0103] It is a comprehensive and standard practice to use multiple evaluation indicators such as NDCG (Normalized Discounted Cumulative Gain), MRR (Mean Reciprocal Rank), MAP (Mean Average Precision) to evaluate the ranking effect of graph neural networks and other baseline models. The following is a brief description of these indicators and how to apply them for comparison:
[0104] NDCG (Normalized Discounted Cumulative Gain) takes into account the relevance score at each position in the sorted results and gives higher weight to the top-ranked results. It penalizes the correct items ranked lower by calculating the cumulative gain and applying a discount factor (usually log(position+1)), and then normalizes the results so that the perfectly sorted list scores 1. NDCG is suitable for evaluating the accuracy of Top-K recommendations, especially for evaluating results with differences in rank or importance.
[0105] MRR (Mean Reciprocal Rank) focuses on the position where the correct or relevant items are first found, and is calculated as the average of the reciprocal of the first hit positions of all queries.
[0106] MAP (Mean Average Precision) measures the average precision, that is, for each query, it calculates the mean precision of all relevant items when they are recalled in the predicted order, and then takes the average of all queries. It takes into account both precision and recall, and is a good indicator for evaluating the overall performance in ranking tasks.
[0107] In the embodiment, the experimental design process is mainly described:
[0108] Step 1: Dataset Preparation
[0109] The dataset is Pull Request data crawled from Github. Our dataset contains 15 projects in total: Kubernetes, Cmssw, TensorFlow, Rails, Rust, Symfony, Pandas, React, Salt, Scikit-Learn, Yii2, Terraform, Moby, Django, OpenCV. The amount of PR data for each project is between 10,000 and 70,000, providing enough data samples for model training.
[0110] Step 2: Baseline
[0111] We selected 8 models used in previous studies as our baselines: RankBoost, AdaRank, CA, ListNet, RF, FIFO, SSF, and MCF.
[0112] Step 3: Evaluation Metrics
[0113] NDCG@K, MRR@K, and MAP@K are used as evaluation indicators. K means considering the influence of the first K ranking elements. Different projects determine the K value based on the amount of project data and the number of PRs opened daily.
[0114] Step 4: Experimental results
[0115] The experimental results are shown in Table 1, Table 2, and Table 3. P1-P15 represent projects Kubernetes, Cmssw, TensorFlow, Rails, Rust, Symfony, Pandas, React, Salt, Scikit-Learn, Yii2, Terraform, Moby, Django, and OpenCV, respectively.
[0116] Table 1 shows the calculation results of the indicator NDCG.
[0117] Table 1 NDCG index calculation results
[0118]
[0119] Table 2 shows the calculation results of the indicator MRR.
[0120] Table 2 MRR indicator calculation results
[0121]
[0122]
[0123] Table 3 shows the calculation results of the indicator MAP.
[0124] Table 3 MAP index calculation results
[0125]
[0126] It can be found that in our data set, the evaluation index obtained by calculating the PR priority based on the graph neural network model is the best. It can be proved that in actual scenarios, the present invention has a better PR sorting effect and can help reviewers prioritize the best PR for review.
[0127] It should be noted that the above embodiments are not intended to limit the protection scope of the present invention, and equivalent changes or substitutions made on the basis of the above technical solutions all fall within the protection scope of the claims of the present invention.
Claims
1. A Pull Request priority sorting method based on graph neural network, characterized in that: Sorting based on PR data features and providing higher priority PRs to reviewers includes the following steps: S1: Crawl PR data of 15 large projects from Github, including PR creation time, closing time, and PR modified content length data; S2: Processing of PR data, including processing of null and infinite values, data standardization, automatic assignment of default values, and calculation of label values. S3: Establish a baseline model, and use the baseline model, including the random forest model, to calculate the corresponding ranking results; S4: Establish the edge structure, establish four edge types: "author-author", "reviewer-reviewer", "author-reviewer", and "file-file", and construct the basic structure of the graph; S5: Train the graph neural network model. Based on the created edge structure, train the weight parameters of the graph neural network model, and finally use the graph neural network to calculate the sorting result. S6: Use multiple evaluation indicators such as NDCG, MRR, and MAP to evaluate the ranking results, and finally compare the ranking effects of graph neural networks and various baselines.
2. According to the PullRequest priority sorting method based on graph neural network according to claim 1, it is characterized in that: Crawl PR data, including selecting appropriate projects from the Github code platform and crawling them. The S1 steps are as follows: S1-1: Project selection, choose the right project and ensure that the PR data volume is sufficient. There are many projects on the Github platform. S1-2: Use GitHub API to build a request: GitHub API programmatically accesses repository data, including PRs. The base URL is https: / / api.github.com / repos / {owner} / {repo} / pulls, where {owner} and {repo} are replaced with the username and repository name of the target repository, respectively. Initiate a request: Use a programming language and library (Python's requests library) to send a GET request to the URL built above, and include an access token in the request header for authentication. S1-3: Data parsing and storage, Parsing response: Extract the required information based on the returned JSON data, including PR number, creator, title, status, number of comments, merge status, Exception handling: Add exception handling logic to deal with request failure, timeout or API restriction, Data cleaning: Clean the extracted data and handle format inconsistencies.
3. According to the PullRequest priority sorting method based on graph neural network according to claim 1, it is characterized in that: To process the PR data, the steps are as follows: S2-1: Abnormal data processing, first identify and process the null values in the data, and use advanced filling strategies to maintain the integrity and meaning of the data. In addition, for the processing of infinite values, choose a suitable filling strategy and use the average value for replacement. S2-2: Data standardization. In order to improve the efficiency of model training and ensure the comparability between different features, the processed data is standardized. The main purpose of standardization is to scale the feature values. Min-Max Scaling is selected to map all feature values to the [0,1] interval, making it suitable for the standardization method of graph neural networks to optimize model performance. S2-3: Calculate label features. The calculation of label features involves calculating data such as whether the PR has been merged, the PR life cycle, and the number of PR review rounds as evaluation indicators, and further divides the granularity. Whether the PR is merged represents the value of the PR. If the PR is merged, it means that the PR has contributed to the project library, otherwise it has no contribution. If the PR is merged, the corresponding priority score is 4. If it is not merged, the priority score is 0. The PR lifecycle is the time span from the creation to the closure of a PR. The shorter the time span, the lower the review cost of the PR, which is more conducive to the reviewer's priority review. Each PR is sorted in ascending order according to its lifecycle, and divided into five priority scores of 0, 1, 2, 3, and 4 according to the time interval in which it is located. The number of PR review rounds is the total number of reviews for the PR, which represents the complexity and controversy of the PR. The more rounds, the higher the review cost. Each PR is sorted in ascending order according to the number of review rounds, and divided into five priority scores of 0, 1, 2, 3, and 4 according to the range of the rounds it is in.
4. According to the Pull Request prioritization method based on graph neural network in claim 1, it is characterized in that S3: establishing a baseline model, the steps are as follows: S3-1: Clarify task requirements and goals: First, clearly define the research questions and goals. The ranking here is quantified as priority scores or levels, so the common models of classification tasks or regression tasks are used. S3-2: Literature review: Review the literature in related fields to understand which baseline models are commonly used in similar tasks. S3-3: Generality and simplicity of the model: Select some general and simple models as the baseline. Select baseline models: RankBoost, AdaRank, CA, ListNet, RF, FIFO, SSF, MCF.
5. The PullRequest priority sorting method based on graph neural network according to claim 1 is characterized in that: S4: Establish the edge structure, as follows: S4-1: "Author-Author" Edges: If two authors have submitted a PR together, an edge is established between them. The weight of the edge reflects the frequency of cooperation or the success rate of cooperation. S4-2: Reviewer-Reviewer Edges: Similarly, if reviewers often review PRs together, an edge is established, and the weight reflects the closeness of cooperation or the degree of consensus. S4-3: "Author-Reviewer Edges": When a specific author's PRs are frequently reviewed by a specific reviewer, an edge is established, and the weight is the historical success rate or speed of their collaboration. S4-4: "File-File" Edges: Edges are established between PRs that have modified the same file, and the weight is the degree of association between files or the similarity of modifications.
6. The PR priority sorting based on graph neural network according to claim 1 is characterized in that: In S5, based on the created edge structure, the weight parameters of the graph neural network model are trained, and finally the sorting result is calculated using the graph neural network. Here are the steps: S5-1: Based on the previously calculated node feature data and node edge data, input data into the model. S5-2: Perform calculations on K-layer networks. The input of each layer depends on the output of the previous layer. S5-3: In the network calculation process of each layer, traverse each node v, integrate the state information of all neighboring nodes of the node, calculate the importance weight a of node v, and combine the state vector and weight parameter to calculate the context information of node v, and pass through a GRU structure to obtain the output of the state vector of node v at the kth layer. S5-4: For the output of the last layer, add a linear output layer to calculate the final priority score. S5-5: According to the list-wise strategy, based on the difference between the predicted score and the true score, the model parameters are adjusted and the model is tuned for multiple epochs.
7. A terminal, characterized in that: It includes a processor, a memory, and a program of a Pull Request priority sorting method based on a graph neural network stored in the memory. When the program of the Pull Request priority sorting method based on a graph neural network is executed by the processor, the steps of the Pull Request priority sorting method based on a graph neural network as described in any one of claims 1-6 are implemented.
8. A computer-readable storage medium having computer instructions stored thereon, characterized in that: When the computer instruction is executed by a processor, the steps of the Pull Request priority sorting method based on graph neural network as described in any one of claims 1 to 6 are implemented.
