A method for predicting a pull request decision result based on early features

Abstract

The application discloses a method for predicting the decision result of a pull request based on early features. The method obtains specific early manual features from the developer features, the pull request entity features and the project features, and obtains the deep semantic features of the pull request from the description and the code changes of the pull request based on a deep learning method. Based on the existing research results on the pull request features, the method combines the deep learning technology, and represents the important description information and the code change information in the pull request as a multi-dimensional space vector through a pre-training model, and plays an important role in the prediction of the pull request result. The application can predict the decision result of the pull request at an early stage of the creation of the pull request, which is beneficial to reducing the burden of the project integrator in maintaining the open source project, and can provide feedback to the creator of the pull request as soon as possible.

Description

Technical Field

[0001] This technology relates to the field of automated code collaboration, and in particular to an automatic prediction method for the decision-making results of pull requests in collaborative code development. Background Technology

[0002] The pull-based development model is a collaborative mechanism in distributed software development and is widely used in open-source software development practices. In this model, external developers, also known as contributors, replicate the project's base repository and independently complete their code changes. Once the code changes are ready, the contributor can submit a pull request to the base repository. Project integrators and other developers can review the code changes in the pull request and interact with the developers through comments and other means. Ultimately, the project maintainers decide whether to merge the pull request based on its quality and the discussions surrounding it.

[0003] In 2021, GitHub reported that its platform generated 170 million merge pull requests. However, reviewing pull requests requires a significant amount of effort from project integrators, becoming a heavy burden as the number of pull requests increases. Our collected data shows that the average time from pull request creation to closure exceeds 37 days. To reduce the workload for project integrators, researchers have built predictive models to predict whether a pull request will be merged. They extract features from different aspects of pull requests, along with other relevant information, such as code changes, the number of comments, and developer experience. Some researchers have found that whether a pull request modifies recently modified code is a crucial factor influencing the decision, while others have found that random forest models based on 14 features perform admirably in predicting whether a pull request will be merged. Recently, researchers, after compiling nearly 10 years of pull request data, established a mixed-effects logistic regression model to empirically study pull requests. They found that a small number of features can explain the decision outcome of pull requests, with whether the creator and integrator of the pull request are the same person being a significant influencing factor. However, most relevant predictive models use "advanced" features, meaning features that don't exist when the pull request is created, such as the number of comments in the pull request. We believe such predictive models are not very helpful in practice because project integrators still need to check the content of new pull requests and communicate with developers after they are created. This doesn't reduce the workload of project integrators. Predicting whether a pull request can be merged at the time of its creation is one possible optimization approach. This is because project integrators can get immediate feedback and optimize the review order of different pull requests based on their workload. Summary of the Invention

[0004] This invention addresses the shortcomings of existing technologies by proposing a method for predicting pull request decision outcomes based on early features. This method allows for the prediction of the final merge result when a pull request is created. A typical use case for this invention is providing decision-making suggestions to project integrators when reviewing pull requests.

[0005] This invention is achieved through the following technical solution:

[0006] A method for predicting fetch request decision results based on early features, specifically:

[0007] Extract title and description information from the pull request, and extract code change information from the pull request's commit log; the code change information consists of newly added and deleted code segments.

[0008] Extract a set of handcrafted features based on the contributor's historical activity information, the entity information of the pull request, and / or the project information of the pull request;

[0009] The title and description information are concatenated and fed into the first feature extraction model to calculate the first multidimensional spatial vector representation R1 of the pull request description content; the code change information is fed into the second feature extraction model to calculate the second multidimensional spatial vector representation R2 of the code change information; the first multidimensional spatial vector representation R1 and the second multidimensional spatial vector representation R2 are concatenated and passed through a fully connected layer to obtain the third multidimensional spatial vector R3 representing the semantic information of the pull request;

[0010] The obtained third multidimensional space vector representation R3 and the handcrafted features are fed into the classifier, and the predicted decision result is output.

[0011] The first feature extraction model, the second feature extraction model, and the classifier are trained based on the collected training dataset.

[0012] This invention proposes a novel method to improve the efficiency of collaborative code development by automatically predicting the decision outcome of a pull request early in its creation. It is based on two parts of feature information: (1) deep semantic features calculated based on the pull request description and code change content; and (2) early manual features based on the developer dimension, pull request entity dimension, and project dimension. By combining deep learning technology to understand the semantic information of pull requests and predicting their decision outcome early in their creation, this method helps reduce the burden on project integrators when reviewing pull requests and also helps external participants obtain timely feedback, thereby improving the efficiency of collaborative software development.

[0013] Furthermore, the extraction of title and description information from the fetch request specifically includes:

[0014] Retrieve the title text and description text from the fetch request;

[0015] The title and description texts are processed and extracted to obtain title and description information respectively. The processing and extraction methods include: retaining the content before the first occurrence of "---" in the text and discarding the text content after "---"; discarding the text lines that start with "-[]"; deleting the web page comments in the text and replacing the code blocks, commit hash values ​​and web page addresses in the text with fixed characters respectively.

[0016] Furthermore, the manual features extracted based on the contributors' historical activity information for pull requests include: whether the contributor to the pull request is a bot account; the number of projects created, the number of commits created, the number of pull requests created, the number of issues created, the number of followers, and the number of projects with commits; within the project containing the current pull request, the number of issues created by the contributor, the number of issues created within the past month, the number of issues participated in historically, and the number of issues participated in within the past month; within the project containing the current pull request, the number of pull requests created by the contributor, the number of merged pull requests, and the number of closed pull requests, and the ratio of merged pull requests to created pull requests; one or more of these features.

[0017] The handcrafted features extracted from the entity information of the pull request include: the length of the current pull request description; whether the description length is greater than the average length of historical pull requests; whether the description length is greater than the median length of historical pull requests; whether the current pull request is the first pull request created by this contributor; the number of early commits contained in the current pull request; the total number, maximum, and minimum number of lines added in the early commits contained in the current pull request; the total number, maximum, and minimum number of lines deleted in the early commits contained in the current pull request; the total number, maximum, and minimum number of lines modified in the early commits contained in the current pull request; the number of files modified by the current pull request; whether the current pull request existed before the first commit was created; and whether the modifications in the current pull request include one or more of the following: test files and document files.

[0018] The manual features extracted from the project information of pull requests include: the number of historical pull request merges, closes, and merge ratios in the project; the number of early commits in the project's historical pull requests, and the number of early commits in pull requests created within the last month; the number of historical pull requests in the project and the number of pull requests created within the last month; the number of issues in the project and the number of issues created within the last month; the number of comments in the project's historical pull requests and the number of comments in pull requests created within the last month; and one or more of the following: the number of comments in the project's historical issues and the number of comments in issues created within the last month.

[0019] Furthermore, the first feature extraction model is a VDCNN, BERT, or RoBERTa model.

[0020] Furthermore, the second feature extraction model is a VDCNN, BERT, or CodeBERT model.

[0021] Furthermore, the classifier is a logistic regression model, decision tree, random forest, or XGBoost classifier.

[0022] Furthermore, each sample in the training dataset includes title information, description information, code change information, handcrafted features, and decision labels that correspond one-to-one with the pull request; the decision labels are merge or reject.

[0023] Furthermore, the decision labels are obtained based on the following method:

[0024] If the pull request meets the following five conditions, the decision label is "Merge"; otherwise, it is "Reject":

[0025] 1. The "merged_at" field in the pull request information obtained from the GitHub API contains specific time information;

[0026] 2. In the historical activity information of the pull request, there is an event with a status of "approved" that has not been revoked;

[0027] 3. All commits included in the pull request are included in the project's default branch;

[0028] 4. The last three comments of the pull request contain the keywords "merged", "landed", "pushed", "included", "committed", "applied", "pulled", and "integrated", and are immediately followed by commit information belonging to the project's default branch;

[0029] 5. The pull request is mentioned or closed in the commit log of the project's default branch.

[0030] Furthermore, the first feature extraction model, the second feature extraction model, and the classifier are trained based on the collected training dataset, and the specific training method is as follows:

[0031] Each sample in the training dataset is used as input to the pre-trained first feature extraction model and the second feature extraction model to obtain the third multidimensional space vector R3. Then, an additional fully connected layer is used to generate the prediction result. With the goal of minimizing the error loss function between the prediction result and the decision label of the sample, the parameters of the first feature extraction model and the second feature extraction model are fine-tuned to obtain the trained first feature extraction model and the second feature extraction model.

[0032] Then, each sample in the training dataset is used as the input to the trained first feature extraction model and the second feature extraction model. The prediction result is obtained through the classifier. With the goal of minimizing the error loss function between the prediction result and the decision label of the sample, the parameters of the classifier are fine-tuned, and finally the trained first feature extraction model, the second feature extraction model and the classifier are obtained.

[0033] Furthermore, the error loss function between the classifier's output and the sample's decision label adopts one or more of the following: binary logistic loss, binary squared loss, and binary cross-entropy loss.

[0034] The beneficial effects of this invention are:

[0035] Currently, there is no method to predict the decision outcome of a pull request solely based on features generated at the time of request creation. This invention, building upon existing research on pull request features, uses deep learning technology to represent crucial descriptive and code change information within the pull request as multi-dimensional vectors through a pre-trained model. This representation plays a significant role in predicting the pull request's decision outcome. This invention can predict the decision outcome early in the pull request's creation process, reducing the burden on project integrators maintaining open-source projects and providing faster feedback to the pull request creator. Attached Figure Description

[0036] Figure 1 This is a flowchart of the present invention;

[0037] Figure 2 This is a demonstration diagram of a deep learning-based encoder; Detailed Implementation

[0038] This invention predicts the decision outcome of a pull request by utilizing feature information present at the time of request creation, including developer features, pull request entity features, project features, and deep semantic features representing descriptions and code changes using deep learning technology. The invention will be further described below with reference to the accompanying drawings.

[0039] A method for predicting pull request decision results based on early features. This invention aims to achieve the task of predicting the decision results of pull requests. This method is also known as E-PRedictor. (Reference) Figure 1 , 2 The overall flowchart of E-PRedictor's decision-making process is presented. Contributors submit pull requests to the project repository. After the pull request is created, manual feature extraction and deep text feature extraction are performed separately. The manual features and deep text features are then concatenated and used as input to the classifier. The classifier outputs the predicted decision result. Specifically, the process includes the following steps:

[0040] S1. Extract title and description information from the pull request, and extract code change information from the pull request's commit log;

[0041] This step primarily involves extracting relevant information from the current pull request to subsequently utilize the encoder model to construct deep semantic features representing code changes. Specifically, this includes:

[0042] The title text t and description text m are obtained from the fetch request, and then processed and extracted to obtain the title information t′ and description information m′ respectively. The text processing and extraction process is as follows: The content before the first occurrence of "---" in the text is retained, while the text content after "---" is discarded. Lines of text starting with "-[]" are discarded. Based on the regular expressions "(<!--.*?-->)" and "( <details> .*？< / details> The text is processed by removing webpage comments. The code blocks in the text are replaced with fixed characters such as "code-block" using the regular expression "```[^\S\r\n]*[az]*\n.*? \n```". The commit hash values ​​in the text are replaced with fixed characters such as "commit-sha1" using the regular expression "\b[0-9a-f]{7}\b|\b[0-9a-f]{40}\b". Finally, the webpage addresses in the text are replaced with fixed characters such as "gh-url". This text processing is repeated to retrieve the title and description information from the request, resulting in the final title information t′ and description information m′.

[0043] Retrieve code change information `c` from the commit log of the pull request, and after processing, obtain the code change information `c′` used by the method. The processing is as follows: Filter the code text information based on the filename extension of the file containing the code changes, retaining text change information ending with "py", "java", "js", "ts", "dart", "php", "rb", "go", "c", "cpp", "cc", "sass", "sh", etc. Concatenate the added code lines to form the "new code segment", and concatenate the deleted code lines to form the "deleted code segment". Concatenate the "new code segment" and "deleted code segment" to obtain the code change information `c′`.

[0044] S2. Extract a set of handcrafted features based on the contributor's historical activity information, the entity information of the pull request, and / or the project information of the pull request;

[0045] This step primarily involves creating manual characteristics using information already present when the pull request is created, including contributor characteristics, pull request entity characteristics, and project characteristics. Specifically, it includes the following sub-steps:

[0046] S2.1 Based on the contributor's historical activity information in the pull request, create one or more of the following 15 handcrafted features f1 to f15 related to them:

[0047] Determine if an account is a bot account by checking if the login name ends with "bot" (f1). Obtain the number of projects created by the contributor when creating the pull request (f2), the number of commits created (f3), the number of pull requests created (f4), the number of issues created (f5), the number of followers (f6), and the number of projects with commits (f7). Within the project containing the current pull request, obtain the number of issues the contributor has created (f8), the number of issues created within the past month (f9), the number of historical issue discussions participated in (f10), and the number of issue discussions participated in within the past month (f11). Additionally, calculate the number of pull requests created by the contributor (f12), the number of merged pull requests (f13), and the number of closed pull requests (f14) within the project containing the pull request, and calculate the ratio of merged pull requests to created pull requests (f15).

[0048] S2.2. Based on the entity information retrieved, create one or more of the 18 related handcrafted features f16 to f33, as follows:

[0049] The pull request is evaluated based on three characteristics: the length of the current pull request description (f16), whether the description length is greater than the average length of historical pull requests (f17), and whether the description length is greater than the median length of historical pull requests (f18). It is also evaluated whether this is the first pull request created by this contributor (f19). The pull request contains the number of earlier commits (f20). Nine characteristics are considered: the total number of lines added in these commits (f21), the maximum value (f22), and the minimum value (f23); the total number of lines deleted in these commits (f24), the maximum value (f25), and the minimum value (f26); and the total number of lines modified in these commits (f27), the maximum value (f28), and the minimum value (f29). The number of files modified in the pull request (f30); whether the pull request existed before the first commit was created (f31); and whether the modifications in the pull request include test files (f32) and document files (f33).

[0050] Based on the project information retrieved from the request, create one or more of the 13 related manual features f34 to f46, as follows:

[0051] The project includes the following: number of merged historical pull requests (f34), number of closed pull requests (f35), and merge ratio (f36). The project also includes the number of early commits in historical pull requests (f37) and the number of early commits in pull requests created within the last month (f38). The project further includes the number of historical pull requests (f39) and the number of pull requests created within the last month (f40). The project also includes the number of issues (f41) and the number of issues created within the last month (f42). The project also includes the number of comments in historical pull requests (f43) and the number of comments in pull requests created within the last month (f44). Finally, the project includes the number of comments in historical issues (f45) and the number of comments in issues created within the last month (f46).

[0052] Each pull request can contain a maximum of 46 early handcrafted features.

[0053] S3. Based on the information obtained in step S1, construct deep semantic features representing the description and code changes using an encoder model, specifically as follows: Concatenate the title information t′ and description information m′ and feed them into the first feature extraction model to calculate the first multidimensional spatial vector representation R1 of the pull request description content. Feed the code change information c′ into the second feature extraction model to calculate the second multidimensional spatial vector representation R2. Concatenate the first multidimensional spatial vector representation R1 and the second multidimensional spatial vector representation R2, and pass them through a fully connected layer to obtain the third multidimensional spatial vector R3 representing the semantic information of the pull request.

[0054] S4. Input the obtained third multidimensional space vector representation R3 and the hand-crafted features into the classifier and output the predicted decision result.

[0055] In steps S3 and S4, the first feature extraction model can be VDCNN, BERT, RoBERTa, etc., and the second feature extraction model can be VDCNN, BERT, CodeBERT, etc. The classifier can be Logistic Regression, Decision Tree, Random Forest, XGBoost, etc. For the description information of the pull request, RoBERTa is preferred for generating its encoded features because it is a more robust pre-trained model based on BERT. For the code change information of the pull request, CodeBERT is preferred for generating its encoded features because it exhibits higher performance on code-related tasks compared to BERT. Furthermore, based on the collected training dataset, the first and second feature extraction models are fine-tuned to map the description and code information of the pull request into a deep semantic vector space, enabling the final classifier to accurately output the decision result.

[0056] Each sample in the training dataset includes a title, description, code change information, handcrafted features, and a decision label corresponding to a pull request. The decision label represents the true decision result of the pull request, which is either merge or reject. Specifically, based on a heuristic conditional judgment method, the true decision result is obtained from the information of closed pull request activities. The conditions for determining that the true decision result of a pull request is "merge" are as follows:

[0057] 1. The "merged_at" field in the pull request information obtained from the GitHub API contains specific time information.

[0058] 2. In the historical activity information of the pull request, there are events with a status of "approved" that have not been revoked.

[0059] 3. All commits included in the pull request are included in the project's default branch.

[0060] 4. The last three comments of the pull request contain keywords such as "merged", "landed", "pushed", "included", "committed", "applied", "pulled", and "integrated", and are immediately followed by commit information belonging to the project's default branch.

[0061] 5. The pull request is mentioned or closed in the commit log of the project's default branch.

[0062] If none of the above conditions are met, then the actual decision result of the pull request is considered to be "rejection".

[0063] To train and fine-tune the pre-trained language model, the actual decision results (merge, reject) of pull requests are used as the training objective during the fine-tuning process. Specifically, each sample in the training dataset is used as input to the pre-trained first and second feature extraction models. The description information and code change information of the pull request are used to generate two 768-dimensional vectors, R1 and R2, respectively, through the encoding model. These two vectors are concatenated and passed to a fully connected layer to produce a 30-dimensional vector representation, R3. R3 is then passed through an additional fully connected layer to produce a 2-dimensional vector as the prediction result. The parameters of the first and second feature extraction models are fine-tuned with the goal of minimizing the error loss function between the prediction result and the decision label of the sample, resulting in the trained first and second feature extraction models. Each sample in the training dataset is then used as input to the trained first and second feature extraction models, and the prediction result is obtained through a classifier. The parameters of the classifier are fine-tuned with the goal of minimizing the error loss function between the prediction result and the decision label of the sample, finally resulting in the trained first and second feature extraction models and the classifier. The loss function can be binary logistic loss, binary squared loss, binary cross-entropy loss, etc., with binary cross-entropy loss being preferred. The trained first feature extraction model, second feature extraction model, and classifier can then be used for prediction in steps S1-S4.

[0064] Additionally, it's important to note that the decision score output by the XGBoost classifier for pull requests can be used to determine the final decision result by setting a threshold, such as a threshold score of 0.5. If the score is greater than the threshold, the pull request is predicted as merged; otherwise, it is predicted as rejected.

[0065] This invention, based on existing research on pull request characteristics, uses deep learning technology to represent the more important descriptive and code change information in pull requests as multi-dimensional space vectors through a pre-trained model, enabling accurate prediction of pull request decision outcomes. This invention can reduce the workload of project integrators when facing a large number of pull requests. The prediction results of this method serve as supplementary information, increasing the information available to external contributors and allowing them to modify and optimize pull requests before the project integrator responds.

[0066] To illustrate the effectiveness of the E-PRedictor method, this embodiment uses ten-fold cross-validation to evaluate the model (the first feature extraction model is the RoBERTa model, the second feature extraction model is the CodeBERT model, and the classifier is the XGBoost classifier). In ten-fold cross-validation, the collected dataset is randomly divided into ten parts. Nine of these parts are used as training data for model training, using the binary cross-entropy loss function, while the remaining part is used as test data to evaluate the model's performance. The class distribution of the data remains consistent across the training data, test data, and original data. To evaluate the effectiveness of this invention, the metrics used include accuracy%, precision@Merge% based on pooling, recall@Merge% and F1 score F1@Merge%, precision@Reject% based on rejection, recall@Reject% and F1 score F1@Reject%, and AUC value AUC%. Table 1 illustrates the average results of the model of this invention in ten-fold cross-validation for different evaluation metrics. E-Predictor outperforms other baseline models in accuracy, pooled precision, F1 score, rejection precision, F1 score, and AUC. Compared to the XGBoost method, which uses only handcrafted features, E-Predictor performs better on AUC equivalence, indicating that semantic features generated by deep learning play a role in prediction.

[0067] Furthermore, a time-order-based cross-validation method was used to evaluate these models. Pull requests in the dataset were sorted by closing time and divided into ten non-overlapping portions. The first n portions were used to train the model, while the (n+1)th portion was used to evaluate model performance. Table 2 illustrates the average results of the models on different evaluation metrics in ten-fold time-order cross-validation. E-PRedictor outperformed the other baseline models in terms of accuracy, pooled precision and F1, rejection-based precision, recall and F1, and AUC.

[0068] Table 1. Evaluation results of 10-fold randomized cross-validation for the pull request decision prediction task:

[0069]

[0070] Table 2. Evaluation results of tenfold time-order cross-validation for the pull request decision prediction task:

[0071]

[0072] Obviously, the above embodiments are merely illustrative examples for clear explanation and are not intended to limit the implementation. Those skilled in the art will recognize that other variations or modifications can be made based on the above description. It is neither necessary nor possible to exhaustively list all possible implementations here. However, obvious variations or modifications derived therefrom are still within the scope of protection of this invention.

Claims

1. A method for predicting fetch request decision results based on early features, characterized in that, Specifically: Extract title and description information from the pull request, and extract code change information from the pull request's commit log; the code change information consists of newly added and deleted code segments. Extract a set of handcrafted features based on the contributor's historical activity information, the entity information of the pull request, and / or the project information of the pull request; The title and description information are concatenated and fed into the first feature extraction model to calculate the first multidimensional spatial vector representation R1 of the pull request description content; the code change information is fed into the second feature extraction model to calculate the second multidimensional spatial vector representation R2 of the code change information; the first multidimensional spatial vector representation R1 and the second multidimensional spatial vector representation R2 are concatenated and passed through a fully connected layer to obtain the third multidimensional spatial vector R3 representing the semantic information of the pull request; The obtained third multidimensional space vector representation R3 and the handcrafted features are fed into the classifier, and the predicted decision result is output. The first feature extraction model, the second feature extraction model, and the classifier are trained based on the collected training dataset.

2. The method according to claim 1, characterized in that, The extraction of title and description information from the fetch request specifically involves: Retrieve the title text and description text from the fetch request; The title content text and description content text are sorted and extracted to obtain title information and description information respectively. The sorting and extraction methods include: keeping the content before the first appearance of "---" in the text and discarding the text content after "---"; discarding the text lines that start with "-[]" in the text; deleting the web page comments in the text and replacing the code blocks, commit hash values ​​and web page addresses in the text with fixed characters respectively.

3. The method according to claim 1, characterized in that, The manual features extracted from the contributor's historical activity information for pull requests include: whether the contributor to the pull request is a bot account; the number of projects, commits, pull requests, issues, followers, and projects with commits made by the contributor when creating the pull request; the number of issues created by the contributor, the number of issues created within the past month, the number of issues participated in historically, and the number of issues participated in within the past month in the project containing the current pull request; and one or more of the following in the project containing the current pull request: the number of pull requests created by the contributor, the number of merged pull requests, and the number of closed pull requests; and the ratio of merged pull requests to created pull requests. The handcrafted features extracted from the entity information of the pull request include: the length of the current pull request description; whether the description length is greater than the average length of historical pull requests; whether the description length is greater than the median length of historical pull requests; whether the current pull request is the first pull request created by this contributor; the number of early commits contained in the current pull request; the total number, maximum, and minimum number of lines added in the early commits contained in the current pull request; the total number, maximum, and minimum number of lines deleted in the early commits contained in the current pull request; the total number, maximum, and minimum number of lines modified in the early commits contained in the current pull request; the number of files modified by the current pull request; whether the current pull request existed before the first commit was created; and whether the modifications in the current pull request include one or more of the following: test files and document files. The manual features extracted from the project information of the pull request include the number of historical pull request merges, the number of closed pull requests, and the merge ratio contained in the project; the number of early commits contained in the historical pull requests of the project; and the number of early commits contained in pull requests created within the last month. The project's historical pull requests and the number of pull requests created within the past month; the project's issues and the number of issues created within the past month; the number of comments contained in the project's historical pull requests and the number of comments contained in pull requests within the past month; and one or more of the following: the number of comments contained in the project's historical issues and the number of comments contained in issues created within the past month.

4. The method according to claim 1, characterized in that, The first feature extraction model is VDCNN, BERT, or RoBERTa.

5. The method according to claim 1, characterized in that, The second feature extraction model is VDCNN, BERT, or CodeBERT.

6. The method according to claim 1, characterized in that, The classifier is a logistic regression model, decision tree, random forest, or XGBoost classifier.

7. The method according to claim 1, characterized in that, Each sample in the training dataset includes a title, description, code change information, handcrafted features, and a decision label that corresponds one-to-one with the pull request; the decision label is either merge or reject.

8. The method according to claim 7, characterized in that, Decision labels are obtained based on the following method: If the pull request meets the following five conditions, the decision label is "Merge"; otherwise, it is "Reject":

1. The "merged_at" field in the pull request information obtained from the GitHub API contains specific time information; 2. In the historical activity information of the pull request, there is an event with a status of "approved" that has not been revoked; 3. All commits included in the pull request are included in the project's default branch; 4. The last three comments of the pull request contain the keywords "merged", "landed", "pushed", "included", "committed", "applied", "pulled", and "integrated", and are immediately followed by commit information belonging to the project's default branch; 5. The pull request is mentioned or closed in the commit log of the project's default branch.

9. The method according to claim 1, characterized in that, The first feature extraction model, the second feature extraction model, and the classifier are trained based on the collected training dataset. The specific training method is as follows: Each sample in the training dataset is used as input to the pre-trained first feature extraction model and the second feature extraction model to obtain the third multidimensional space vector R3. Then, an additional fully connected layer is used to generate the prediction result. With the goal of minimizing the error loss function between the prediction result and the decision label of the sample, the parameters of the first feature extraction model and the second feature extraction model are fine-tuned to obtain the trained first feature extraction model and the second feature extraction model. Then, each sample in the training dataset is used as the input to the trained first feature extraction model and the second feature extraction model. The prediction result is obtained through the classifier. With the goal of minimizing the error loss function between the prediction result and the decision label of the sample, the parameters of the classifier are fine-tuned, and finally the trained first feature extraction model, the second feature extraction model and the classifier are obtained.

10. The method according to claim 1, characterized in that, The error loss function between the classifier's output and the sample's decision label uses one or more of the following: binary logistic loss, binary squared loss, and binary cross-entropy loss.

Images