A Software Defect Feature Selection Method Based on a Two-Stage Hybrid of ReliefF and CFS

Abstract

The application provides a software defect feature selection method based on two-stage mixing of ReliefF and CFS, and belongs to the field of software defect prediction.The method combines weight and correlation, and is realized based on ReliefF algorithm and CFS algorithm, so that the features with high correlation and low redundancy in data can be selected.The method comprises the following steps: extracting the metric element features of a software product to be predicted, calculating the correlation between each metric element feature and a feature category by using the ReliefF algorithm, removing irrelevant features according to a set threshold, then calculating the correlation between the remaining features and the category and sorting the features, and finally screening the features according to the heuristic equation of CFS to remove redundant features, so that the features valuable for software defect prediction are obtained.The method can select the features with high correlation and low redundancy in software defect data, and then be used for high-accuracy prediction of software defects, and can be used for further optimization of the performance of subsequent software defect prediction.

Description

TECHNICAL FIELD

[0001] The application belongs to the field of software defect prediction, and particularly relates to a software defect feature selection method based on two-stage mixing of ReliefF and CFS. BACKGROUND

[0002] Feature selection is an important preprocessing step in machine learning, which has been widely used in bioinformatics, biological networks and text classification and many other fields. Kira et al. proposed a filter feature selection method Relief, which randomly selects m instances from the training dataset. For each selected instance i, Relief calculates the nearest neighbor of the same class i and the nearest neighbor of the opposite class. According to whether the features distinguish the two instances from the same class or different classes, the quality of each feature is estimated. The earliest proposed Relief algorithm is mainly for binary classification problem. The algorithm is simple, efficient and effective, and has been widely used. However, it is limited to processing two-class problems. Therefore, Kononenko extended it in 1994 and obtained the ReliefF algorithm. ReliefF is an extension of Relief for handling multi-classification problems. It directly solves the multi-classification problem. ReliefF algorithm is used to handle the regression problem with continuous value of target attribute. Its main idea is to take the Euclidean distance as the correlation index, then weight the features according to the distinguishing degree of different classes, find the nearest neighbor of each class of the current sample and calculate it comprehensively. At the same time, the features are selected in the process of model training by embedding method, and the feature selection result is output after the training is completed. However, the ReliefF algorithm also has some defects: first, it does not consider the effect of multiple features in classification, second, it cannot remove redundant features.

[0003] Hall proposed a correlation-based feature selection algorithm CFS, which is a classical filter feature selection method. It is a heuristic algorithm for evaluating the value or advantage of feature subset. This heuristic considers the usefulness of each feature for predicting the class label and the degree of correlation between them. CFS first calculates the feature class and feature correlation matrix according to the training set data, and then searches the feature subset space by best-first search. Forward selection, backward elimination and other search methods can also be used. CFS algorithm evaluates the effect of single feature on each classification heuristically, and obtains the final feature subset. Although CFS algorithm has good dimensionality reduction ability, the solution obtained by it is not necessarily a global optimal solution.

[0004] A hybrid feature selection method proposed by Shivkumar et al. When the feature quantity is reduced to a certain extent, the algorithm predicts that the performance begins to decline due to the lack of important information. The algorithm combines the advantages of filter method and wrapper method, but has high time complexity. Guo et al. proposed a software defect prediction method based on random forest, and the prediction accuracy of the method is higher than that of logistic regression and discriminant analysis. Compared with other methods, random forest has stronger robustness to noise and outliers of defect data set. However, the irrelevant features have a great influence on random forest, and the effect of training random forest using the top 5 most relevant features is equivalent to that of training using all features.

[0005] The defects of the existing feature selection algorithms can be summarized as follows: the filter-based feature selection algorithm can remove irrelevant features, but cannot remove redundant features. The feature selection algorithm based on embedded method combines the advantages of filter and wrapper, and has high complexity problem. At present, most of the feature selection algorithms are only one of removing irrelevant features or redundant features, and cannot effectively select the features with high correlation and low redundancy in the data set at the same time. If the feature selection method of software defect prediction selects software defect feature data with high redundancy and low correlation, the data value is low, which affects the accuracy of subsequent software defect prediction. SUMMARY

[0006] The present application is directed to the problem that the features with low redundancy and high correlation in the data set cannot be effectively selected at the same time, and a new method is proposed: a software defect feature selection method based on two-stage hybrid of ReliefF and CFS, so as to select the features with high correlation and low redundancy in the software defect data.

[0007] In order to achieve the above purpose, the technical scheme adopted by the present application is as follows:

[0008] A software defect feature selection method based on two-stage hybrid of ReliefF and CFS, comprising the following steps:

[0009] Step 1, extracting the metric element features of the software product to be predicted;

[0010] Step 2, calculating the correlation feature weight of the metric element features and the feature categories based on ReliefF;

[0011] Step 3, setting a threshold to remove irrelevant features to obtain a feature subset F1;

[0012] Step 4, calculating the correlation rel between the remaining features in the feature subset F1 and the classes;

[0013] Step 5, the software defect feature subset F1 obtained in step 3 is sorted in descending order according to the correlation rel, to obtain a sorted software defect feature subset F1';

[0014] Step 6, the heuristic rule Merit based on CFS s The correlation of the software defect feature subset F1' is evaluated, and a forward selection strategy is used to screen the software defect feature subset, to obtain a final software defect feature subset F2, which is a feature valuable for software defect prediction.

[0015] Further, the metric features include LOC count, Halstead complexity, and McCabe complexity.

[0016] Further, the specific process of step 2 is as follows:

[0017] Step 2.1, first calculate the sample center point P of each feature category sample Cent , the calculation formula is as follows:

[0018] E = ∑ |p i -p o | (1)

[0019] Where p i represents any sample point in a certain category of samples, p o is an initially randomly selected non-center point, ΔE represents the error function value after p i is replaced by p o ; if ΔE < 0, it indicates that the error will decrease after replacement, then p i is replaced by p o , that is, p i is taken as the new sample center point; otherwise, it is not replaced; in this way, the sample center point P Cent of each category is finally determined.

[0020] Step 2.2, calculate the Euclidean distance D iCent of the sample points in each category sample and the center point of the sample of the category, the calculation formula is as follows:

[0021] D iCent = ||p i -p Cent || (2)

[0022] Where p i represents any sample point in a certain category of samples, p Cent represents the sample center point of a certain category of samples.

[0023] Step 2.3: Divide the samples of each category into groups, and select the center sample of each group as the final selected sample P; the specific process is as follows:

[0024] The samples are divided into g = m / c groups, where m is the number of ReliefF samplings and c is the number of classes in the dataset. The Euclidean distance between the groups is Δd = [Max(D icent )-Min(D icent Finally, the center sample of each group of samples is selected.

[0025] Step 2.4: Using the selected sample P, ReliefF retrieves the k nearest neighbors of P from the same class, denoted as nearHits, and retrieves the k nearest neighbors of P from each different class, denoted as nearMisses. If the difference between P and nearHits for a metric feature is less than the difference between P and nearMisses, then this is a feature that is beneficial for classification, and the weight of that metric feature is increased; otherwise, the weight of that metric feature is decreased. This process is repeated until the termination condition is met. Finally, the weights of the metric features are returned. The formula for calculating the feature weight W is:

[0026]

[0027] Where f represents the feature, P represents the sample, and hit j This represents the k nearest neighbors found in the same class of samples, and diff(·) represents the k nearest neighbors found in sample P and sample hit. j The difference on feature f, where Class(·) represents the sample class, p(·) represents the probability, and miss... j (C) represents the j-th nearest neighbor sample in class C.

[0028] Furthermore, in step 3, the threshold δ is set as the median, and its calculation formula is as follows:

[0029] δ=mid(W) (4)

[0030] Where W represents the feature weight.

[0031] Furthermore, the specific process of step 4 is as follows:

[0032] Step 4.1: Define `rel` to combine the feature weights of each feature to calculate the correlation between each feature and the category. The calculation formula is as follows:

[0033] rel=R cf +W-2R cf ·W / (R cf +W) (5)

[0034] Among them, R cfThe formula representing the association between feature f and category C is as follows:

[0035] R cf =SU(f,C) (6)

[0036] Where SU(·) represents the symmetric uncertainty function, which measures the correlation between features through symmetric uncertainty. Its formula is as follows.

[0037]

[0038] Where H(X) and H(Y) represent the entropy of discrete random variables X and Y, respectively. Assuming x represents every possible value of variable X, the entropy of X is calculated as follows:

[0039]

[0040] Where p(x) represents the probability that X takes the value x;

[0041] IG(X,Y) represents the information gain function, used to measure the reduction in entropy of X given a value of Y. It describes the degree of change in entropy of X after a given value of Y, reflecting the additional information provided by feature Y to X. The larger the value of IG(X,Y), the higher the correlation between X and Y. Its calculation method is as follows.

[0042] IG(X,Y)=H(X)-H(X|Y) (9)

[0043] Where H(X|Y) is the conditional entropy, which is defined as:

[0044]

[0045] Where p(y) represents the probability that Y takes the value y, and p(x|y) represents the probability that x takes the value y given y.

[0046] The information gain is normalized to [0,1] using the symmetric uncertainty method to ensure that the attributes can be compared with each other and that different choices produce the same effect. When its value is 1, it means that the knowledge of X can completely predict the knowledge of Y. When its value is 0, it means that X and Y are independent of each other.

[0047] Furthermore, the specific process of step 6 is as follows:

[0048] A forward selection strategy is used to filter the software defect feature subset F1'. The feature corresponding to the maximum value in the relevance rel is selected and added to the software defect feature subset F2. Merit is then calculated. s Merit s The evaluation method is calculated as follows:

[0049]

[0050] Merit s This represents the evaluation of a feature subset S containing t features; The average correlation between features and categories; The average correlation between features measures the redundancy among features, and its formula is as follows:

[0051]

[0052]

[0053] Where t represents the number of features, f represents the sample features, C represents the sample category, and i and j both represent the ordinal numbers of the features;

[0054] If new software defect features are added to Merit s If the feature increases, keep the current feature in feature subset F2 and continue adding the remaining features. If the feature remains unchanged or decreases, remove it from feature subset F2 and continue adding other features until all features have been traversed. Finally, output the filtered feature subset, which has removed redundant features and contains the features that are valuable for predicting software defects.

[0055] The beneficial technical effects of this invention are as follows:

[0056] 1. To address the unfairness in weight calculation caused by random sample selection, a method of uniformly selecting samples from each category is proposed, which yields stable weights. Furthermore, it avoids situations where certain categories are difficult to select due to insufficient sample size, thus preventing their participation in the software defect prediction process.

[0057] 2. To address the inaccuracy in removing irrelevant features due to threshold settings, setting the threshold to the median will allow for more representative features to be used in software defect prediction.

[0058] 3. To address the issue that considering only weights while ignoring the correlation between features and categories may result in many irrelevant features remaining in the filtered feature subset, we propose using a combination of weights and correlation to calculate the correlation between the remaining features and the class. This approach can more comprehensively determine the distinguishing effect of features on the class, thereby selecting features that are more valuable for software defect prediction and improving the classification accuracy of the software defect prediction model. Attached Figure Description

[0059] Figure 1 The flowchart shows the software defect feature selection method based on a two-stage hybrid of ReliefF and CFS according to the present invention.

[0060] Figure 2 In the experiment of this invention A comparison of the average AUC of different feature selection algorithms under the Bayes classifier;

[0061] Figure 3 In the experiment of this invention A comparison of the average F-values ​​of different feature selection algorithms under the Bayes classifier. Detailed Implementation

[0062] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments:

[0063] The method of this invention adopts a scheme that combines weighting and correlation, and is implemented based on the ReliefF algorithm and the CFS algorithm. The purpose is to select features with high correlation and low redundancy in the data, and then use them for high-accuracy prediction of software defects.

[0064] like Figure 1 As shown, a software defect feature selection method based on a two-stage hybrid of ReliefF and CFS includes the following steps:

[0065] Step 1: Extract the metric features of the software product to be predicted:

[0066] The metric features include LOC count, Halstead complexity, and McCabe complexity, all of which are stored in ARFF (Attribute-Relational File Format).

[0067] Step 2: Calculate the relevance feature weights between the metric features and feature categories based on ReliefF. The specific process is as follows:

[0068] Step 2.1: First, calculate the sample center point P for each feature category sample. Cent The calculation formula is as follows:

[0069] E=∑|p i -p o | (1)

[0070] Where, p i p represents any sample point in a certain class of samples. o Let p be an initially randomly selected non-center point, denoted by ΔE. i Replace p o The error function values ​​before and after substitution. If ΔE < 0, it means the error will decrease after substitution, then p is used. i Replace p o p i Use it as the new sample center point; otherwise, do not replace it. In this way, the sample center point P for each category is finally determined. Cent .

[0071] Step 2.2: Calculate the Euclidean distance D between each sample point in each class and the center point of that class. iCent The calculation formula is as follows:

[0072] D iCent =||p i -p Cent || (2)

[0073] Where, p i p represents any sample point in a certain class of samples. Cent It represents the center point of a sample in a certain class.

[0074] Step 2.3: Divide the samples of each category into groups, and select the center sample of each group as the final selected sample P; the specific process is as follows:

[0075] The samples are divided into g = m / c groups, where m is the number of ReliefF samplings and c is the number of classes in the dataset. The Euclidean distance between the groups is Δd = [Max(D icent )-Min(D icent Finally, the center sample of each group of samples is selected.

[0076] Step 2.4: Using the selected sample P, ReliefF retrieves the k nearest neighbors of P from the same class, denoted as nearHits, and retrieves the k nearest neighbors of P from each different class, denoted as nearMisses. If the difference between P and nearHits for a given metric feature is less than the difference between P and nearMisses, then this is a feature beneficial for classification, and the weight of that metric feature is increased. Conversely, the weight of that metric feature is decreased. This process is repeated until the termination condition is met, and finally, the weight of the metric feature is returned, which represents the correlation between the metric feature and the target class. The formula for calculating the feature weight W is:

[0077]

[0078] Where f represents the feature, P represents the sample, and hit j This represents the k nearest neighbors found in the same class of samples, and diff(·) represents the k nearest neighbors found in sample P and sample hit. j The difference on feature f, where Class(·) represents the sample class, p(·) represents the probability, and miss... j (C) represents the j-th nearest neighbor sample in class C.

[0079] Step 3: Set a threshold to remove irrelevant features and obtain the feature subset F1.

[0080] When setting a threshold to remove irrelevant features, the median is chosen as the threshold δ because it is unaffected by excessively large or small values ​​and represents the general level of the entire data, thus having good representativeness. Its calculation formula is as follows:

[0081] δ=mid(W) (4)

[0082] Where W represents the feature weight.

[0083] Step 4: Calculate the correlation rel between the remaining features in feature subset F1 and the class. The specific process is as follows:

[0084] Step 4.1: Define `rel` to combine the feature weights of each feature to calculate the correlation between each feature and the category, thus providing a more comprehensive assessment of the feature's ability to distinguish between categories. The calculation formula is as follows:

[0085] rel=R cf +W-2R cf ·W / (R cf +W) (5)

[0086] Among them, R cf The formula representing the association between feature f and category C is as follows:

[0087] R cf =SU(f,C) (6)

[0088] Where SU(·) represents the symmetric uncertainty function, which measures the correlation between features through symmetric uncertainty. Its formula is as follows.

[0089]

[0090] Where H(X) and H(Y) represent the entropy of discrete random variables X and Y, respectively. Assuming x represents every possible value of variable X, the entropy of X is calculated as follows:

[0091]

[0092] Where p(x) represents the probability that X takes the value x;

[0093] IG(X,Y) represents the information gain function, which measures the decrease in entropy of X given a value of Y. It describes the degree of change in entropy of X after a given value of Y, and can reflect the additional information provided by feature Y to X. The larger the value of IG(X,Y), the higher the correlation between X and Y. Its calculation method is as follows.

[0094] IG(X,Y)=H(X)-H(X|Y) (9)

[0095] Where H(XY) is the conditional entropy, which is defined as:

[0096]

[0097] Where p(y) represents the probability that Y takes the value y, and p(x|y) represents the probability that x takes the value y given y.

[0098] By normalizing the information gain to [0,1] using the symmetric uncertainty method, we can ensure that the attributes are comparable and that different choices produce the same effect. A value of 1 indicates that knowledge of X can completely predict knowledge of Y. A value of 0 indicates that X and Y are independent of each other.

[0099] Step 5: Sort the software defect feature subset F1 obtained in Step 3 in descending order according to the relevance rel to obtain the sorted software defect feature subset F1'.

[0100] Step 6: Merit based on CFS heuristic rules s The relevance of the software defect feature subset F1' is evaluated, and a forward selection strategy is used to filter the software defect feature subset to obtain the final software defect feature subset F2 and return it, thus obtaining features that are valuable for software defect prediction.

[0101] A forward selection strategy is used to filter the software defect feature subset F1'. The feature corresponding to the maximum value in the relevance rel is selected and added to the software defect feature subset F2. Merit is then calculated. s Merit s The evaluation method is calculated as follows:

[0102]

[0103] Merit s This represents the evaluation of a feature subset S containing t features; The average correlation between features and categories; The average correlation between features measures the redundancy among features, and its formula is as follows:

[0104]

[0105]

[0106] Where t represents the number of features, f represents the sample features, and C represents the sample category. i and j both represent the ordinal numbers of the features.

[0107] If new software defect features are added to Merit sIf the feature size increases, it remains in feature subset F2. The remaining features are then added. If the size remains unchanged or decreases, the feature is removed from feature subset F2. Other features are added, and this process continues until all features have been traversed. Finally, the filtered feature subset is output, which has been freed of redundant features and contains only the features valuable for predicting software defects.

[0108] This invention underwent verification and comparative experiments. The specific steps of the verification experiments are as follows:

[0109] Experimental Environment Configuration: The development platform used in the experiment was PyCharm 2018.3.2, and the software environment was Python 3.7.2. The hardware environment used by the program was a computer with 16GB RAM, an Intel Core i7-6700HQ CPU@2.60GHz processor, and a 256GB solid-state drive.

[0110] The experiment selected 10 software defect datasets from the NASA MDP database. Table 1 provides a summary of these datasets based on language, number of instances, number of modules, number of defective modules, and proportion of defective modules. Specific information is shown in Table 1.

[0111] Table 1 Dataset Information

[0112]

[0113]

[0114] like Figure 1 As shown, the logical process of the algorithm of this invention in finding the optimal subset of software defect features is as follows:

[0115] Input: Original dataset OrData, original software defect feature set F, number of sample samplings m, number of nearest neighbor samples k

[0116] Output: FeSubset software defect feature subset

[0117] Step 1: Initialize feature weights W = {w i {i = 1, 2, ..., M}; T is the empty set.

[0118] Step 2: Calculate the Euclidean distance D between the center point of each class and all other sample points. iCent .

[0119] Step 3: According to D iCent The size divides this type of sample into g groups, with the Euclidean distance between the groups being Δd.

[0120] Step 4: Traverse each target feature class sample and select the intermediate sample P from each group of samples in that class.

[0121] Step 5: Find k nearest neighbor samples that are closest to P from both the same class and different class sample sets. j (j = 1, 2, ..., k); , miss j (j = 1, 2, ..., k;)

[0122] Step 6: Update the weights of each feature

[0123] Step 7: Set the threshold δ = mid(W(A))

[0124] Step 8: Compare the weight of each feature with the threshold. If the weight of a feature is less than the threshold, then remove that feature.

[0125] Step 9: Using weights and R cf Combining computational features with class correlation rel

[0126] Step 10: Sort F in descending order of weight rel.

[0127] Step 11: Initialize the merit of the feature subset to -∞

[0128] Step 12; sequentially select the features f i Add to FeSubset

[0129] Step 13: Using Merit s Calculate the relevance assessment value of this feature subset. If merit ≥ merit + Merit s Then remove the feature from FeSubset.

[0130] Step 14: Return FeSubset

[0131] This invention employs an M×N fold cross-validation method to divide the feature subset into a training set and a test set, utilizing typical... The Bayes classification algorithm is used to build a defect prediction model on the training set to verify its feasibility and effectiveness. To address the class imbalance problem in the dataset, this invention uses 10×5-fold cross-validation, dividing the dataset into a test set and a training set according to a 5-fold cross-validation method. The training set is used for model training, and the test set is used to evaluate the trained model. This process is repeated 10 times, and the average of these 10 results is taken as the final evaluation result. To verify the merits of the RCH method in this invention, it is compared with the traditional ReliefF algorithm, CFS algorithm, and the RFC algorithm proposed by Xu X et al. The results are as follows: Figures 2-3 As shown in Table 2.

[0132] Table 2 is based on Comparison of different Bayes classification algorithms

[0133]

[0134] As can be seen from Table 2, based on On the same dataset, the Bayes classification algorithm proposed in this invention improves the average AUC by 7.3% compared to the traditional ReliefF method, 5.9% compared to the CFS method, and 2.9% compared to the RFC method. The average F-value is improved by 16.5% compared to the traditional ReliefF method, 3.1% compared to the CFS method, and 4.2% compared to the RFC method.

[0135] Therefore, it can be concluded that the method proposed in this invention has a significant effect on improving the performance of software defect prediction, achieving good results in removing redundant and irrelevant features. This invention can better select features with low redundancy and high correlation in the dataset, further optimizing the performance of software defect prediction.

[0136] Of course, the above description is not intended to limit the present invention, and the present invention is not limited to the examples given above. Any changes, modifications, additions or substitutions made by those skilled in the art within the scope of the present invention should also fall within the protection scope of the present invention.

Claims

1. A software defect feature selection method based on two-stage hybrid of ReliefF and CFS, characterized in that, Comprising the following steps: Step 1, extracting metric element features of the software product to be predicted; Step 2, calculating the correlation feature weight of the metric element features and the feature categories based on ReliefF; The specific process is as follows: Step 2.1, first calculate the sample center point of each feature category sample The calculation formula is as follows: (1) wherein, represents an arbitrary sample point in a certain class of samples, is an initial randomly selected non-central point, and represents instead of the error function value after and before replacement; if , it indicates that the error will be reduced after replacement, and is replaced by , that is, is taken as the new sample center point; otherwise, it is not replaced; in this way, the sample center point of each class is finally determined; Step 2.2, calculate the Euclidean distance between each sample point and the center point of the sample of the category The calculation formula is as follows: (2) wherein, represents a sample center point of a certain class of samples; Step 2.3, dividing each category sample into groups, and selecting the center sample of each group sample as the final selected sample P; the specific process is as follows: The samples are divided into groups, wherein is the ReliefF sampling times, is the number of classes in the data set, and the group interval Euclidean distance size is The center sample of each group is finally selected. Step 2.4: Through the selected samples ReliefF retrieves from the same class of The nearest neighbors, denoted as nearHits, are retrieved from each distinct class. of The nearest neighbors, denoted as nearMisses; if on a metric feature If the difference between the feature weight and the feature weight is less than the difference between the feature weight and the feature weight, then this is a feature that is beneficial for classification, and the weight of this feature weight is increased; otherwise, the weight of this feature weight is decreased. This process is repeated until the termination condition is met. Finally, the weights of the feature weights are returned. The formula for calculating the feature weight W is: (3) wherein, represents a feature, represents a sample, represents the nearest neighbor found in the same class of samples, represents a sample and a sample difference in the feature represents a sample class, represents a probability, represents the nearest neighbor sample in the same class;​ Step 3, setting a threshold to remove irrelevant features to obtain a feature subset F1; Step 4, compute the correlation between the remaining features in the feature subset Fl and the classes ; Step 5, the software defect feature subset F1 obtained in step 3 is sorted in descending order according to the correlation to obtain a sorted software defect feature subset F1'; Step 6, CFS-based heuristic rules The correlation of the software defect feature subset F1' is evaluated, and a forward selection strategy is used to screen the software defect feature subset, to obtain a final software defect feature subset F2. The software defect feature subset F2 is a feature valuable for software defect prediction.

2. The method according to claim 1, wherein the two-stage hybrid software defect feature selection method based on ReliefF and CFS is characterized in that, The metric element features include LOC count, Halstead complexity and McCabe complexity.

3. The method of claim 1, wherein the two-stage hybrid software defect feature selection method based on ReliefF and CFS is characterized by, In step 3, the threshold value is set to the median, which is calculated as: (4) Wherein, W represents the feature weight.

4. The method of claim 1, wherein the two-stage hybrid software defect feature selection method based on ReliefF and CFS is characterized by, The specific process of step 4 is as follows: Step 4.1, Definition The relevance between each feature and the category is calculated by combining the feature weight of each feature, and the formula is: (5) wherein, representing features and categories between the categories, which is formulated as follows: (6) Wherein, SU(·) represents a symmetric uncertainty function, which measures the correlation between features by symmetric uncertainty, and its formula is as follows, (7) where and denote the entropy of the discrete random variable assuming for each possible value of the variable , respectively, the entropy of is calculated as (8) wherein denotes taken probability; represents the information gain function, which measures the reduction in entropy of when is given a value, describes the degree of change in entropy of after is given a value, reflects the additional information provided by to The greater the value of , the higher the degree of correlation between , and its calculation method is as follows,​ (9) wherein is the conditional entropy, which is defined as: (10) wherein denotes taken the probability that denotes taken the probability that The information gain is normalized to [0, 1] by symmetric uncertainty method, which ensures that the attributes can be compared with each other and different choices have the same effect; when the value is 1, it means that the understanding of can completely predict the understanding of ; when the value is 0, it means that and are independent of each other.

5. The method of claim 1, wherein the two-stage hybrid software defect feature selection method based on ReliefF and CFS is characterized by, The specific process of step 6 is as follows: The software defect feature subset F1' is screened by using a forward selection strategy, and the features corresponding to the maximum correlation value in the feature set F1 are selected and added to the software defect feature subset F2, and the , evaluation method is calculated as follows: (11) wherein, represents an evaluation of a feature subset S containing features; is the average correlation between a feature and a class; is the average correlation between a feature and a feature, which measures the redundancy between features, and is calculated as follows: (12) (13) wherein, denotes the number of features, denotes the features of the sample, denotes the class of the sample, i, each denotes the serial number of the number of features; If the new software defect feature is added If the new software defect feature is added If the new software defect feature is added If the new software defect feature is added If the new software defect feature is added If the new software defect feature is added If the new software defect feature is added If the new software defect feature is added If the new software defect feature is added If the new software defect feature is added If the new software defect feature is added If the new software defect feature is added If the new software defect feature is added If the new software defect feature is added If the new software defect feature is added If the new software defect feature is added If the new software defect feature is added If the new software defect feature is added If the new software defect feature is added If the new software defect feature is added If the new software defect feature is added If the new software defect feature is added If the new software defect feature is added If the new software defect feature is added If the new software defect feature is added If the

Images