A developer recommendation method based on GitHub social technology network

Abstract

This invention provides a developer recommendation method based on the GitHub social technology network. The method comprises: obtaining behavioral data from the GitHub social technology network and performing basic data cleaning; vectorizing developer nodes in the social network based on the behavioral data; training a named developer value measurement model; receiving a vector of a designated developer node input by a user, calculating the value of the remaining developer nodes in the social technology network where the node resides according to the developer value measurement model; and recommending the top-N candidate developers by value to the user. Compared with existing technologies, this invention improves the accuracy of the developer value measurement model, provides more accurate value assessments of developers in open source projects on the GitHub platform, enhances the quality of recommendation results, and improves the reference value of developer recommendations in the open source field.

Description

Technical Field

[0001] The present invention relates to the fields of open source software ecosystem and developer recommendation technology, and in particular to a developer recommendation method based on the GitHub social technology network. Background Art

[0002] In recent years, with the rapid rise of the open source ecosystem, open source has become a trend in the software industry. Open source software has become a vital component of modern society's digital infrastructure, and its maintenance and sustainable, healthy development are crucial. The continuous development of open source software has gradually formed a distributed collaboration model, and collaborative development platforms based on version control systems (such as GitHub) have become the standard collaboration platform for modern open source software. All users can follow the development of any open source project in the community in real time, submit unlimited merge requests to the software projects they are interested in, and freely comment on other developers' contributions. In every open source project, there are core developers. Core developers contribute significantly to the project and have a significant influence on it. For ordinary developers interested in a specific project, it is important to connect with core developers as much as possible to understand the project's development and trends. However, current methods for measuring the value of developers within a project are still imperfect, making it difficult for ordinary developers to accurately track the core information of the project and participate in open source projects.

[0003] In summary, in order to address the problems of complex developer value measurement and inaccurate recommendations, it is of great significance to establish a complete developer value measurement model and introduce comprehensive indicators for developer recommendation. Summary of the Invention

[0004] This invention addresses the shortcomings of existing technologies by proposing a developer recommendation method based on the GitHub social technology network. This method employs a link prediction model to construct a developer value measurement model based on the social and technical attributes of the developer collaboration network within a project to obtain recommendations for developers within a project. This method effectively addresses the issues of developer value measurement and recommending core developers for open source projects within the open source software ecosystem. This method can help ordinary developers integrate into the open source community more quickly, reduce the cost of understanding core project information, and significantly contribute to the long-term development of open source projects.

[0005] The specific technical solution for achieving the purpose of the present invention is:

[0006] A developer recommendation method based on the GitHub social technology network is characterized by building a relatively comprehensive value measurement system for open source community developers based on the developer social collaboration network and proposing comprehensive network evaluation indicators. Its construction includes the following specific steps:

[0007] Step 1: Obtain behavioral data from the GitHub social technology network and perform basic data cleaning

[0008] We crawled the developer log behavior data from the code repository on GitHub. The crawled behavior data was messy due to various missing or other reasons. We needed to clean the data, delete the unusable data with missing attributes, and process it into a neat, standardized, and easy-to-read dataset.

[0009] Step 2: Vectorized representation of developer nodes

[0010] The developer behavior data obtained in step 1 is divided into two categories based on behavior type: social behavior data between developers and technical behavior data between developers and the code repository. The two types of behavior data contain different log data types. Seven types of behavior data from the two categories are selected to vectorize the developer nodes in the repository.

[0011] Step 3: Build a developer value measurement model

[0012] For the vector data representing the behavioral characteristics of developer nodes obtained in step 2, a social technology network is constructed based on the interaction between developers and repositories and the collaborative relationship between developers. Differentiated weights are assigned to the edges in the network. Based on the constructed social technology network, a developer value measurement model is established and trained to obtain the value of developer nodes in the network.

[0013] Step 4: Build and train the developer recommendation model

[0014] Based on homogeneous network characteristics, the metric model in step 3 is applied to the developer nodes in the dataset to construct a developer recommendation network. Based on the developer node value obtained in step 3, the link prediction algorithm is used to predict the collaborative relationship between developers. The dataset obtained in step 1 is used to train and validate the model.

[0015] Step 5: Developer Recommendation Model Usage

[0016] After the developer recommendation model is trained in step 4, the vectorized representation of the developer node behavior data obtained in step 2 is input into the obtained developer recommendation model. Based on the developer value measurement results, the top N developers are determined from all developers in the warehouse as the recommendation results; where N is a positive integer.

[0017] The developer node vector model described in step 2 is constructed according to the following steps:

[0018] 2-1: The dataset contains a large amount of log data from Github projects. We filter the data from the dataset by project name to find the corresponding data for the projects to which the developer recommendation method is to be applied.

[0019] 2-2: We select seven types of social and technical behavior data, which all developers have the authority to operate, including following (Star), creating a copy (Fork), commenting on an issue (IssueComment), creating an issue (OpenIssue), creating a code submission (OpenPullRequest), reviewing a code submission (PullRequestReviewComment), and merging a code submission (PullRequestMerged), to construct a vectorized representation of the nodes.

[0020] Among them, each time a developer creates an issue in the selected project, it is recorded as 1 OpenIssue; each time a developer comments on an issue, it is recorded as 1 IssueComment; submitting a pull request for the project is recorded as 1 OpenPullRequest; reviewing the pull request and making corresponding suggestions is recorded as 1 PullRequestReviewComment, comments on specific lines of code are recorded as PullRequestReviewComment, and direct comments on the pull request are recorded as IssueComment events; a pull request proposed by the developer is merged, it is recorded as 1 PullRequestMerged; a developer starring a project is recorded as 1 Star; a developer forking a project is recorded as 1 Fork.

[0021] The developer value measurement model described in step 3 is constructed according to the following steps:

[0022] 3-1: Different weights are assigned to the seven types of developer behavior data, with weight values of 2, 1, 3, 4, 2, 1, and 2 respectively. The developer's value measurement is calculated by the following formula (1):

[0023] V u_d =C issue_comment +2C open_issue +3C open_pr +4C review_comment +2C pr_merged +C star +2C fork (1)

[0024] Where: C issue_comment is the number of comments the developer has made on issues in the code repository, C open_issue The number of new issues created by the developer in the code repository, C open_pr The number of times the developer submits new pull requests in the code repository, C review_commentis the number of comments the developer has made on the pull request in the code repository, C pr_merged The number of times the pull request submitted by the developer in the code repository has been merged, C star The developer's attention status to the code repository, only considering the current status. If the developer star the project, it is counted as 1, otherwise it is counted as 0. fork The number of times the developer forked the code repository;

[0025] 3-2: The PullRequestMerged event count is weighted based on the number of lines of code contained in the Pull Request. That is, for a single Pull Request that modifies 100-299 lines of code, the maximum count value is 1, while the count of Pull Requests that modify less than 100 lines or more than 299 lines is reduced accordingly. The PullRequestMerged count is calculated using the following formula (2):

[0026]

[0027] Where loc represents the number of lines of code modified by a single Pull Request.

[0028] The developer recommendation model described in step 4 is constructed and trained according to the following steps:

[0029] 4-1: Collect raw data and clean it into a data structure that can be directly read by the model;

[0030] 4-2: Divide the cleaned dataset into training set, validation set and test set in a ratio of 6:2:2;

[0031] 4-3: The relationship between node intimacy and network information structure is defined by the following formula (3);

[0032]

[0033] Where n represents a node set; n(i) and n(j) are the neighbor node sets of nodes i and j respectively; n(i)∩n(j) is the intersection of the neighbor nodes of i and j, and n(i)∪n(j) represents the union of the neighbor nodes of i and j.

[0034] 4-4: Use node centrality indicators to measure the importance of nodes. Node importance is defined from three aspects:

[0035] Degree centrality is defined by the following formula (4):

[0036]

[0037] Among them, k represents the degree of the node, ki represents the degree of node i;

[0038] Betweenness centrality is defined by the following formula (5), and normalized betweenness centrality is defined by the following formula (6):

[0039]

[0040]

[0041] Among them: g represents the number of shortest paths, g st is the number of shortest paths between nodes s and t, and m represents the number of paths represents the number of nodes i that the shortest path between nodes s and t passes through, and N represents the number of nodes in the network;

[0042] Closeness centrality is defined by the following formula (7):

[0043]

[0044] Where: d represents the distance, d i represents the mean of the shortest distances from node i to all other nodes, d ij is the distance between nodes i and j;

[0045] 4-5: Based on node importance and node intimacy, a weighted developer social technology collaboration network is constructed, and node intimacy and node importance are introduced as measurement indicators.

[0046] Compared to existing technologies, this invention improves the precision of developer value measurement models, enabling more accurate valuations of developers within open source projects, enhancing the quality of recommendation results, and making developer recommendations more relevant within the open source space. Recommending developers within open source projects can foster more collaborative behavior, increase the density of developers' social technology networks, significantly enhance the influence of open source projects, and improve the reputation of developers among their peers, ultimately promoting the development of the open source ecosystem as a whole. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] Figure 1 A diagram for classifying developer behavior;

[0048] Figure 2 It is a flow chart of the present invention. DETAILED DESCRIPTION

[0049] See Figure 2The present invention obtains behavioral data from the GitHub social technology network and performs basic data cleaning; vectorizes the developer nodes in the social network based on the behavioral data; trains a named developer value measurement model; receives a specified developer node vector input by the user, calculates the value of the remaining developer nodes in the social technology network where the node is located according to the developer value measurement model; and recommends candidate developers ranked top-N in value to the user.

[0050] The present invention is further described in detail through the following specific examples.

[0051] Example 1

[0052] See Figure 2 , follow the steps below to recommend developers based on the GitHub social technology network, including:

[0053] S1: Obtain developer behavior data and perform data cleaning

[0054] The mirror data provided by GHArchive was selected as the data source. The GHArchive project records GitHub's public event stream data and archives the event stream data by hour. Each archived data contains JOSN format data returned by the GitHub API. Users can access these archived data through HTTP. ClickHouse database was selected for data storage. ClickHouse database is a columnar database management system for online analytical processing. Based on specific business and understanding of the open source field, data parsing and cleaning rules were formulated, and the data was stored in the ClickHouse database to provide query and analysis services. In this case, it only takes a few minutes to complete an aggregate query and calculation of the entire domain log data for a whole year using the ClickHouse database, which greatly reduces the time and space complexity of data processing and calculation.

[0055] Check the data integrity in the ClickHouse database and delete any records with missing data in the Star, Fork, IssueComment, OpenIssue, OpenPullRequest, PullRequestReviewComment, and PullRequestMerged fields.

[0056] S2: Vectorized representation of developer nodes in the network

[0057] See Figure 1, read the open source project logs, select the seven types of social and technical behavior data of developers, including Star, Fork, IssueComment, OpenIssue, OpenPullRequest, PullRequestReviewComment and PullRequestMerged. When a developer raises an issue in the project, it is recorded as 1 OpenIssue; each comment in the issue is recorded as 1 IssueComment; submitting a pull request for the project is recorded as 1 OpenPullRequest; reviewing the pull request and making corresponding suggestions is recorded as 1 PullRequestReviewComment, comments on specific code lines are recorded as ReviewComment, and direct comments on pull requests are recorded as IssueComment events; a pull request raised by a developer is merged, which is recorded as 1 PullRequestMerged; a developer stars the project, which is recorded as 1 Star; a developer forks the project, which is recorded as 1 Fork.

[0058] For developers A, B, and C, the vectorized representation of their nodes is shown in the following table:

[0059] ID Star Fork Issue Comment OpenIssue OpenPullRequest PullRequestReviewComment PullRequestMerged A 0 1 23 2 4 2 2 B 1 1 17 13 0 0 0 C 1 1 2 1 7 4 1 D 1 1 4 1 13 6 12 E 1 1 5 0 2 0 1 F 1 1 11 3 6 2 0

[0060] S3: For the input target open source project, calculate the initial developer node value of the candidate node in its network according to formula (1), and correct the node value according to formula (2):

[0061] V u_d =C issue_comment +2C open_issue +3C open_pr +4C review_comment +2C pr_merged +C star +2C fork (1)

[0062]

[0063] The final values of nodes A, B, C, D, E, and F are shown in the following table:

[0064] Node ID Node Value A 52.64 B 46 C 45.776 D 93 E 15.81 F 46

[0065] S4: Based on the project network structure, combined with node value and measurement indicators, the recommended scoring order of AF is D, B, A, C, F, E. The recommended TOP3 developers are D, B, and A.

[0066] The above is only an example of an embodiment of the present invention. The scope of protection of the present invention is not limited to the above example. All technical solutions based on the concept of the present invention are within the scope of protection of the present invention. It should be pointed out that for those skilled in the art, various improvements without departing from the principles of the present invention should be considered as within the scope of protection of the present invention.

Claims

1. A developer recommendation method based on GitHub social technology network, characterized in that: The method comprises the following specific steps: Step 1: Obtain behavioral data from the GitHub social technology network and perform basic data cleaning We crawled the developer log behavior data from the code repository on GitHub. The crawled behavior data was messy due to various missing or other reasons. We needed to clean the data, delete the unusable data with missing attributes, and process it into a neat, standardized, and easy-to-read dataset. Step 2: Vectorized representation of developer nodes The developer behavior data obtained in step 1 is divided into two categories based on behavior type: social behavior data between developers and technical behavior data between developers and code repositories. The two types of behavior data contain different types of log data. Seven types of behavioral data from the two categories are selected to vectorize the developer nodes in the warehouse; Step 3: Build a developer value measurement model For the vector data representing the behavioral characteristics of developer nodes obtained in step 2, a social technology network is constructed based on the interaction between developers and repositories and the collaborative relationship between developers. Differentiated weights are assigned to the edges in the network. Based on the constructed social technology network, a developer value measurement model is established to obtain the value of developer nodes in the network. Step 4: Build and train the developer recommendation model Based on the homogeneous network characteristics, the measurement model in step 3 is applied to the developer nodes in the dataset to build a developer recommendation network; Based on the developer node value obtained in step 3, the collaborative relationship between developers is predicted using the link prediction algorithm. The dataset obtained in step 1 is used to train and verify the model. Step 5: Developer Recommendation Model Usage After the developer recommendation model is trained in step 4, the vectorized representation of the developer node behavior data obtained in step 2 is input into the obtained developer recommendation model. Based on the developer value measurement results, the top N developers are determined from all developers in the warehouse as the recommended results, where N is a positive integer. The developer value measurement model described in step 3 is constructed as follows: 3-1: Different weights are assigned to the seven types of developer behavior data, with weight values of 2, 1, 3, 4, 2, 1, and 2 respectively. The developer's value measurement is calculated by the following formula (1): V u_d =C issue_omment +2C open_issue +3C open_pr +4C review_comment +2C pr_merged +C star +2C fork (1) Where: c issue_omment is the number of comments the developer has made on issues in the code repository, C open_issue The number of new issues created by the developer in the code repository, C open_pr The number of times the developer submits new pull requests in the code repository, C review_comment is the number of comments the developer has made on the pull request in the code repository, C pr_merged The number of times the pull request submitted by the developer in the code repository has been merged, C star The developer's attention status to the code repository, only considering the current status. If the developer star the project, it is counted as 1, otherwise it is counted as 0. fork The number of times the developer forked the code repository; 3-2: The PullRequestMerged event count is weighted based on the number of lines of code contained in the Pull Request. That is, for a single Pull Request that modifies 100-299 lines of code, the maximum count value is 1, while the count of Pull Requests that modify less than 100 lines or more than 299 lines is reduced accordingly. The PullRequestMerged count is calculated using the following formula (2): Where loc represents the number of lines of code modified by a single Pull Request.

2. The developer recommendation method based on GitHub social technology network according to claim 1, characterized in that: The vectorized representation of the developer node described in step 2 is constructed as follows: 2-1: The dataset contains a large amount of log data from Github projects. We filter the data from the dataset by project name to find the corresponding data for the projects to which the developer recommendation method is to be applied. 2-2: Select seven types of social and technical behavior data that all developers in a specified project have permission to perform: Star, Fork, Issue Comment, OpenIssue, OpenPullRequest, PullRequestReviewComment, and PullRequestMerged, and construct a vectorized representation of the nodes. Among them, each time a developer creates an issue in the selected project, it is recorded as 1 OpenIssue; each time a developer comments on an issue, it is recorded as 1 IssueComment; submitting a pull request for the project is recorded as 1 OpenPullRequest; reviewing the pull request and making corresponding suggestions is recorded as 1 PullRequestReviewComment, comments on specific lines of code are recorded as PullRequestReviewComment, and direct comments on the pull request are recorded as IssueComment events; a pull request proposed by the developer is merged, it is recorded as 1 PullRequestMerged; a developer starring a project is recorded as 1 Star; a developer forking a project is recorded as 1 Fork.

3. The developer recommendation method based on GitHub social technology network according to claim 1, characterized in that: The developer recommendation model described in step 4 is constructed and trained according to the following steps: 4-1: Collect raw data and clean it into a data structure that can be directly read by the model; 4-2: Divide the cleaned dataset into training set, validation set and test set in a ratio of 6:2:2; 4-3: The relationship between node intimacy and network information structure is defined by the following formula (3); Where n represents a node set; n(i) and n(j) are the neighbor node sets of nodes i and j respectively; n(i)∩n(j) is the intersection of the neighbor nodes of i and j, and n(i)∪n(j) represents the union of the neighbor nodes of i and j. 4-4: Use node centrality indicators to measure the importance of nodes. Node importance is defined from three aspects: Degree centrality is defined by the following formula (4): Among them, k represents the degree of the node, k i represents the degree of node i; Betweenness centrality is defined by the following formula (5), and normalized betweenness centrality is defined by the following formula (6): Among them: g represents the number of shortest paths, g st is the number of shortest paths between nodes s and t, and m represents the number of paths represents the number of nodes i that the shortest path between nodes s and t passes through, and N represents the number of nodes in the network; Closeness centrality is defined by the following formula (7): Where: d represents the distance, d i represents the mean of the shortest distances from node i to all other nodes, d ij is the distance between nodes i and j; 4-5: Based on node importance and node intimacy, a weighted developer social technology collaboration network is constructed, and node intimacy and node importance are introduced as measurement indicators.

Images