Undersampling and software defect prediction method based on risk perception layering
By using a risk-aware hierarchical undersampling method, combined with SHAP value analysis and hierarchical random sampling, the problems of data distribution distortion and decreased model generalization ability caused by undersampling are solved, and more efficient software defect prediction is achieved.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- HANGZHOU DIANZI UNIV
- Filing Date
- 2026-01-19
- Publication Date
- 2026-05-05
AI Technical Summary
Existing undersampling methods tend to distort data distribution and reduce model generalization ability in software defect prediction, and have high computational complexity, making it difficult to effectively balance the problem of class imbalance.
We employ a risk-aware hierarchical undersampling method, which analyzes feature importance using SHAP values, calculates continuous risk scores for majority class samples, and performs hierarchical random sampling to retain key samples in order to construct a balanced training dataset.
It effectively avoids the blind deletion of high-value samples, maintains the spatial distribution structure of the data, improves the generalization ability and robustness of the model, and significantly improves the prediction performance.
Smart Images
Figure CN121980264A_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the fields of software engineering and artificial intelligence technology, and specifically relates to an undersampling and software defect prediction method based on risk perception layering. Background Technology
[0002] Software Defect Prediction (SDP) is a core task in software engineering, aiming to automatically identify potentially defective modules by analyzing historical data using machine learning models. SDP is not only a key means of ensuring software quality but also an important tool for improving development efficiency, supporting decision-making, and promoting intelligent software engineering. However, in actual software development, defective samples are naturally scarce, while non-defective samples dominate, resulting in a significant class imbalance in the dataset. This imbalanced distribution causes model training to be biased towards the majority class, leading to a decline in prediction performance.
[0003] Existing methods for mitigating class imbalance mainly include data resampling techniques, cost-sensitive learning, and ensemble learning. Cost-sensitive learning penalizes misclassification of the minority class by adjusting the weights of the loss function, but the weights are difficult to quantify precisely under complex distributions. Ensemble learning improves classification performance by combining multiple base learners, but its high computational cost limits its application on large-scale datasets. Data resampling techniques are further divided into oversampling and undersampling. While oversampling increases the proportion of the minority class by synthesizing new samples, it often induces overfitting due to the introduction of artificial noise. Undersampling balances the dataset by removing majority class samples, significantly reducing computational complexity and avoiding the introduction of minority class noise, demonstrating unique application value. However, traditional undersampling methods may distort the original data distribution due to indiscriminate sample removal, leading to a shift in decision boundaries and a decrease in generalization ability. Although some improved methods attempt to optimize undersampling through ranking, problems such as distorted majority class distribution, high computational complexity, and poor interpretability still exist. Therefore, a method for undersampling that can effectively balance class distribution while maintaining data integrity is still needed. Summary of the Invention
[0004] To address the shortcomings of existing technologies, this invention proposes an undersampling and software defect prediction method based on risk perception stratification. It utilizes SHAP values to analyze feature importance and combines them with defect distribution patterns to quantify the potential cost of incorrectly removing each majority class sample. Furthermore, it performs stratified sampling based on risk scores, thereby avoiding blind deletion and distribution distortion.
[0005] An undersampling method based on risk perception hierarchy specifically includes the following steps:
[0006] Step 1: Obtain the software defect prediction dataset, which includes feature vectors and class labels. Based on the sample class labels, divide the non-defect samples into the majority class sample set and the defect samples into the minority class sample set, and perform data preprocessing.
[0007] Step 2: Build a basic classification model, use SHAP (SHapley Additive exPlanations) values to quantify the global importance of each feature in the entire dataset to the prediction results, and remove noisy features based on importance.
[0008] Step 3: For the retained features, perform adaptive binning based on the feature values, calculate the relative defect rate of each bin, and generate bin weights by combining the feature importance quantified by the SHAP value; aggregate the weights of the bins to which each feature belongs for the majority class samples, and calculate the continuous risk score for each majority class sample.
[0009] Step 4: Based on the calculated continuous risk scores, use the quantile binning method to divide all majority class samples into multiple risk levels. Each risk level contains approximately the same number of samples to preserve the spatial distribution characteristics of the samples.
[0010] Step 5: Based on the set balance ratio, calculate the number of samples to be retained in each risk level, and conduct random sampling independently within each level. Combine the sampling results with the minority class sample set to output a balanced software defect prediction dataset.
[0011] A software defect prediction method based on risk perception hierarchical undersampling is proposed. The method described above is used to sample the majority class sample set in the software defect prediction dataset, and then merge it with the original minority class sample set to form a balanced training dataset for training the software defect prediction model. The trained software defect prediction model is then used to predict new samples.
[0012] Compared with the prior art, the present invention has the following beneficial effects:
[0013] 1. This invention transforms the undersampling process into a risk assessment problem. Unlike traditional methods that simply screen based on distance or density, this invention combines feature importance with the distribution of minority class samples to calculate a continuous risk score for each majority class sample. By directly measuring the cost of removing majority class samples, it can identify key samples located near the decision boundary that contain rich discriminative information, thus effectively avoiding the problem of model performance degradation caused by blindly removing high-value samples.
[0014] 2. To address the problem of excessive concentration of remaining samples distorting the original distribution, this invention proposes a risk-level-based random sampling strategy. By dividing the majority class samples into different levels according to risk scores and performing stratified sampling, this ensures that the sampled subset retains high-risk boundary samples while maintaining the representativeness of low-risk samples. This strategy effectively reduces class imbalance while preserving the spatial distribution structure of the original data to the greatest extent, thereby significantly improving the generalization ability and robustness of the prediction model. Attached Figure Description
[0015] Figure 1 Flowchart of a software defect prediction method based on risk perception hierarchical undersampling. Detailed Implementation
[0016] The present invention will be further explained below with reference to the accompanying drawings;
[0017] like Figure 1 As shown, a software defect prediction method based on risk perception hierarchical undersampling firstly performs feature importance analysis, risk assessment, and hierarchical undersampling on the original software defect dataset to construct a balanced training dataset. Then, the software defect prediction model is trained, and finally, defect prediction is performed on unknown samples. The specific steps are as follows:
[0018] Step 1: In this embodiment, 18 publicly available datasets from AEEEM, PROMISE, and NASA repositories are selected as the original software defect datasets. To ensure the generalizability of the experimental results, these publicly available datasets cover projects of different sizes and imbalance rates. For the above datasets, data cleaning is first performed to remove duplicate samples to prevent data leakage.
[0019] Step 2: Perform importance analysis on sample features using a tree ensemble model. In this embodiment, a random forest classifier is trained. The SHAP value is used to quantify the contribution of each feature to defect prediction for each sample in the dataset. Calculate the SHAP values of all its features. Then the feature global importance for:
[0020]
[0021] Where n represents the total number of samples in the dataset. Based on global importance... Sort and filter all features to remove noisy features.
[0022] Step 3: Combine the local defect distribution pattern and the importance of global features to conduct a risk assessment on the majority of samples.
[0023] s3.1 For the features retained in step 2, an adaptive binning strategy is used to divide the feature values into multiple intervals, and the binning calculation is performed for each interval. Actual defect rate and relative defect rate :
[0024]
[0025]
[0026] in, The defect rate of the entire dataset. This represents the total number of minority class samples in the dataset. It is a box The number of minority class samples accounted for a certain percentage of the bins. The proportion of the total sample size. s3.2, Using the Sigmoid function to calculate the relative defect rate. Convert to bins weight :
[0027]
[0028] This step can amplify the weight of high defect rate intervals and accurately identify key feature intervals.
[0029] s3.3, aggregate the binning weights of all features of the sample, and calculate the majority class sample. raw risk score :
[0030]
[0031] in, The number of features to retain in step 2. Indicates sample Features The value of , express The associated binning index, for Packing The corresponding weights.
[0032] Original risk score Normalized by minimum-maximum scaling to The interval is used to obtain the final risk score. .
[0033] Step 4: Use quantile binning based on risk score The majority class sample set Divided into 10 risk levels ,in ~ The risk scores of the samples increase sequentially. This stratification strategy ensures that each bin contains approximately equal sample sizes, thereby reducing imbalance while maintaining the original data distribution structure.
[0034] Step 5: First, set a uniform retention rate for each risk level. ,in This is the number of majority class samples in the entire dataset. Then, for each layer, the number of samples is calculated:
[0035]
[0036] in, This indicates rounding down. This represents the number of samples in the l-th risk level, and max() represents taking the maximum value.
[0037] Finally, random selection is made from each level. Each sample is merged to form a majority class subset. and with a minority set Combine them to construct the final balanced training dataset. .
[0038] The balanced dataset was input into the classifier for training, and five-fold cross-validation was used to evaluate the model performance. To demonstrate the effectiveness of this method, a comparative experiment was conducted with existing techniques, and AUC was chosen as the evaluation metric. The experimental results are shown in Table 1:
[0039] Table 1
[0040]
[0041] Compared with existing technologies, this invention demonstrates superior performance in terms of AUC. Specifically, this method achieves the highest AUC values across multiple datasets, including PC1, PC4, and JDT, reaching a maximum of 0.8416. On an overall average level, the invention achieves an average AUC of 0.6704, significantly outperforming existing technologies. This fully demonstrates that through risk perception and hierarchical sampling strategies, this invention can more effectively capture the decision boundaries of defective patterns than existing technologies, exhibiting stronger robustness and predictive accuracy when dealing with class imbalance problems.
[0042] Furthermore, the experimental results show that while the RUS method performs well in some projects, it does not consider the feature distribution of samples during sampling and randomly removes samples from the majority class, resulting in the accidental deletion of a large number of samples containing key decision-making information and causing serious information loss. The Cluster method uses cluster centers to replace the original samples, which preserves the outline of the data distribution, but often smooths out discriminative edge details. Optimization methods such as LTRUS usually only focus on hard-to-classify or boundary samples, tending to delete samples in all safe regions. While this strategy strengthens the boundaries, it severely distorts the global probability distribution of the original data, making the model prone to overfitting during prediction. However, this method can accurately identify potential defect patterns, thus demonstrating high robustness and efficiency in software projects of different sizes and types.
Claims
1. An undersampling method based on risk perception hierarchies, characterized in that: Construct a basic classification model, use SHAP values to quantify the global importance of each feature in the original software defect prediction dataset to the prediction results, and remove noisy features based on importance; For the retained features, adaptive binning is performed based on the feature values. The weights of each bin are generated by combining the global importance of the features with the proportion of minority class samples in each bin. The weights of the bins to which each feature belongs in the majority class sample are aggregated to calculate the continuous risk score for each majority class sample. Based on the continuous risk scores of the majority class samples, the quantile binning method is used to divide all majority class samples into multiple risk levels with the same sample size. According to the set balance ratio, the same number of majority class samples are independently and randomly sampled in each risk level. The sampling results are then merged with the minority class samples in the original software defect prediction dataset to output a balanced software defect prediction dataset.
2. The undersampling method based on risk perception layering as described in claim 1, characterized in that: The basic classification model is a tree ensemble model.
3. The undersampling method based on risk perception layering as described in claim 1, characterized in that: feature global importance for: Where n represents the total number of samples in the original software defect prediction dataset; Represents the features in the i-th sample The SHAP value.
4. The undersampling method based on risk perception layering as described in claim 3, characterized in that: The kth bin based on eigenvalues weight for: in, , Indicates box division The actual defect rate and the relative defect rate, It is a box The number of minority class samples accounted for a certain percentage of the bins. The proportion of the total sample size; This represents the global defect rate. This represents the total number of minority class samples in the original software defect prediction dataset.
5. The undersampling method based on risk perception layering as described in claim 1, characterized in that: The majority class samples Continuous risk score for: in, The number of features remaining after removing noise features. Indicates sample Chinese characteristics The value of , express The associated binning index, The weights are the corresponding weights for each bin.
6. The undersampling method based on risk perception layering as described in claim 5, characterized in that: right Normalize to the [0,1] interval using minimum-maximum scaling.
7. The undersampling method based on risk perception layering as described in claim 1, characterized in that: Sampling within each risk level 100 samples, of which 100 samples 200 samples 300 samples 400 samples 500 samples 600 samples 700 samples 800 samples 900 samples 1 ... This indicates rounding down. This represents the number of samples in the l-th risk level, and max() represents taking the maximum value; Indicates the balance ratio. , These represent the number of minority class samples and the number of minority class samples in the original software defect prediction dataset, respectively.
8. A computer-readable storage medium having a computer program stored thereon, which, when executed in a computer, causes the computer to perform the method of any one of claims 1 to 7.
9. A software defect prediction method based on risk perception hierarchical undersampling, characterized in that: The majority class samples of the software defect prediction dataset are undersampled using the risk-aware hierarchical undersampling method as described in any one of claims 1 to 7. Then, the majority class samples are merged with the original minority class sample set to form a balanced training dataset, which is used to train the software defect prediction model. The trained software defect prediction model is then used to predict new samples.