Novel data screening method

Abstract

The invention provides a cross-project defect prediction method based on semi-supervised clustering data screening, and aims to effectively improve the performance of a defect prediction model. According to the method, firstly, software module data is processed by utilizing a semi-supervised clustering algorithm, and the data is divided into a plurality of sub-clusters, so that a complex data structure is effectively organized and simplified; then, by analyzing the historical software module category of the current project, screening out cross-project historical software module data consistent with the category label of the historical software module, and forming a screened cross-project data set; and then, combining the screened data with historical software module data of the current project, modeling by using a naive Bayes classification algorithm, and finally constructing a cross-project defect prediction model. In the whole process, the interference of irrelevant data on model training is avoided by screening the relevant data, and meanwhile, the historical software module information of the cross-project and the project is fully fused, so that the actual application effect of defect prediction is enhanced while the accuracy and generalization ability of the model are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of software defect prediction, and in particular, provides a cross-project defect prediction method based on semi-supervised clustering data screening. The method aims to improve the accuracy of cross-project defect prediction, and through data processing and model optimization, effectively solves the problem of limited prediction performance due to data irrelevance in traditional methods. Background Art

[0002] Software defects are potential sources of system errors, failures, and crashes. In order to predict software defects, linear regression data classification algorithms are usually used for testing.

[0003] Linear regression is a statistical method used to model the linear relationship between variables. It predicts the relationship between a dependent variable (target variable) and one or more independent variables (feature variables) by fitting a straight line. Linear regression is widely used in fields such as economics, engineering, and social sciences, especially in data analysis and prediction tasks. The advantages of linear regression include its simplicity, which makes the interpretation and understanding of the model intuitive. At the same time, linear regression is computationally efficient and suitable for processing large-scale data sets. In addition, the linear regression algorithm is easy to implement and can be extended to multivariate linear regression to handle multiple independent variables. Its coefficients are highly interpretable, making it convenient to analyze the relationship between variables.

[0004] However, linear regression also has some disadvantages. First, it assumes that there is a linear relationship between the dependent variable and the independent variable, and has a weaker fitting ability for nonlinear relationships. Linear regression is sensitive to outliers in the data, which may affect the performance of the model and the accuracy of prediction. When there is a high correlation between the independent variables, the stability and interpretability of the model will be affected, resulting in unreliable coefficient estimates. In addition, linear regression requires that the error term follows a normal distribution and has homoscedasticity. If the data does not meet these assumptions, the effectiveness of the model may decrease.

[0005] Aiming at the defects of linear regression, the present invention proposes a new prediction method, which can effectively improve the accuracy of data prediction. Summary of the invention

[0006] The present invention provides a cross-project defect prediction method based on semi-supervised clustering data screening, which aims to improve the accuracy of defect prediction by utilizing cross-project historical data. The method mines the software history warehouse and extracts metric attributes, constructs a mixed training set and trains a naive Bayes prediction model, and finally realizes defect judgment of the software module to be predicted.

[0007] The implementation of this method includes the following steps: first, extract useful software modules and existing modules to be predicted from the software history warehouse of the project and obtain their measurement attributes; then, borrow the cross-project historical software module data to extract the same measurement attributes to form a data set containing 14 cross-project software modules. Next, mark the boundary weights of all software modules and define the minimum radius value for instance density connection to facilitate subsequent semi-supervised clustering.

[0008] Finally, the selected module data and historical software module data are mixed to form a training set to train the naive Bayes model. The model is used to calculate the defective posterior probability of the software module to be predicted and judge its defect status. When the posterior probability meets certain conditions, the module is determined to be defective or non-defective, thereby achieving effective defect prediction. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] Figure 1 A flow chart of cross-project defect prediction based on semi-supervised clustering data screening in an embodiment of the present invention.

[0010] Figure 2 A distribution diagram of the project instance and cross-project instances of an embodiment of the present invention in a 20-dimensional space.

[0011] Figure 3 A diagram showing clustering results of a project instance and cross-project instances according to an embodiment of the present invention. DETAILED DESCRIPTION

[0012] The specific implementation process of the cross-project defect prediction method flow embodiment based on semi-supervised clustering data screening designed by the present invention is as follows: Step 1: Mining the software history repository of this project and extracting useful software modules from it; Step 2, extract the existing software modules to be predicted in this project; Step 3, extract the measurement attributes of the software module of this project; Step 4: When a project is just being developed, there is little historical software module data for the project, so it is not possible to train the defect prediction model for the project. Therefore, it is necessary to borrow cross-project historical software module data; Step 5, extract the same 20 measurement attributes as in step 3 across all software modules in the project; After steps 4-5, this embodiment assumes that 14 cross-project software module data are formed after extracting the measurement attributes and marking class labels. These 14 local project software module data and cross-project software module data can be formed as follows: Figure 2 The distribution diagram in 20-dimensional space is shown; Step 6: Mark all cross-project software modules and all local software modules as unvisited, randomly select an unvisited local historical software module marked as Y or N, assign its boundary weight to 0, and assign the boundary weights of all other unvisited local software modules and all cross-project software modules to infinity. The boundary weight defines the minimum radius value for the density connection between two instances; Step 7: Mix the cross-project software module data selected by the semi-supervised clustering algorithm and the historical software module data of this project to form a mixed training set D , train a naive Bayes prediction model; Step 8: Use the trained prediction model to predict the software module of this project to be predicted X; Step 8.1: Calculate the software modules to be predicted for this project X flawed posterior probabilities; Step 8.2, determine whether the software module to be predicted for this project has defects. P ( X |N)> P ( X |Y), the naive Bayes classification method determines that its class label is N, that is, the software module is a defect-free software module. If P( X |N)< P( X |Y), the naive Bayes classification method determines that its class label is Y, that is, the software module is a defective software module.

Claims

1. A cross-project defect prediction method based on semi-supervised clustering data screening can process software module data and divide the data into multiple sub-clusters through a semi-supervised clustering algorithm. The algorithm combines labeled data and unlabeled data to improve the accuracy of clustering.

2. The method according to claim 1, characterized in that: Filter out data that is consistent with the current project category number from historical software module data to form a filtered cross-project data set.

3. The method according to claim 1, characterized in that: Combine the filtered cross-project data with the historical software module data of the current project to form a comprehensive data set, which provides a basis for subsequent modeling.

4. The method according to claim 1, characterized in that: Using the combined data set, the naive Bayes classification algorithm is applied for modeling to quickly generate a defect prediction model.

5. The method according to claim 4, characterized in that: Evaluate the constructed cross-project defect prediction model and optimize it based on the evaluation results to improve the performance of the model.

Images