An adaptive oversampling method for software defect data based on local density information

Abstract

The present invention discloses an adaptive oversampling method for software defect data based on local density information. The present invention first uses the Min-Max normalization method to unify the dimension in the data preprocessing stage, and uses the proposed Agglomerative Clustering-Relief feature selection method to reduce the data dimension and delete redundant features. Then, in the adaptive group sampling stage, by analyzing the local density information of minority class instances in a low-dimensional state, all minority class instances are divided into different groups according to their respective distribution characteristics, and for minority class instances in different groups, a targeted oversampling strategy is used for adaptive oversampling. Finally, the number of samples of the two types in the data set is balanced, and the imbalance problem is solved from the data level.

Description

Technical Field

[0001] The present invention is a data oversampling method for minority class instances in a class imbalanced data set, aiming to balance the number of two types of samples in a software defect data set after using this technology, thereby improving the classification accuracy of the classifier for defect samples and reducing the cost of software testing. Background Art

[0002] The datasets in the field of software defect prediction naturally have class imbalance problems. The prediction model trained on the class imbalanced dataset will make the prediction results more inclined to the majority class, and the performance of the classifier on the minority class will deteriorate. The minority class represents defective instances, and its prediction performance is more important. Therefore, the class imbalance problem is considered to be one of the main factors affecting the performance of software defect prediction, and more and more researchers are working hard to solve the class imbalance problem in software defect prediction.

[0003] Sample sampling technology, cost-sensitive learning, classification threshold shifting and ensemble learning are currently popular methods for solving class imbalance. Cost-sensitive learning methods assign different misclassification costs to defective and non-defective instances in the data set, but usually require domain prior knowledge to provide a cost matrix, which is not applicable in many real-world problems. In the classification process of the algorithm, the classification threshold shifting method moves the threshold according to the ratio of defective and non-defective samples, and makes predictions on the classifier based on the changed threshold, but it is difficult to establish a quantitative relationship between the threshold parameter and the accuracy of imbalanced classification. Ensemble learning methods combine multiple base classifiers to obtain a classifier with stronger classification performance, which usually has better results, but has a high computational overhead.

[0004] Compared with other methods, sample sampling technology only changes the distribution of data at the data level, is easy to implement, and is independent of the specific classification model. Sample sampling technology is divided into oversampling technology and undersampling technology. Oversampling technology does not lose valuable information in the data, and its effect is usually better than undersampling technology. The SMOTE method is the most widely used oversampling method. According to the shortcomings of the existing SMOTE method, researchers have proposed many improved algorithms based on SMOTE. Although these improved algorithms have made some progress in solving the diversity of samples, fuzzy decision boundaries, and synthetic sample noise problems, they have not fully considered the distribution characteristics of the minority class samples in the data set, resulting in the low quality of the synthesized minority class instances. The classification accuracy of minority class samples still needs to be improved. Summary of the invention

[0005] In view of the deficiencies of the prior art, the present invention proposes an adaptive oversampling method for software defect data based on local density information. The present invention combines the improved method based on the traditional SMOTE oversampling method popular in the industry and the VAE oversampling method used in the oversampling field in recent years. First, the Min-Max normalization method is used to unify the dimensions in the data preprocessing stage, and the proposed Agglomerative Clustering-Relief feature selection method is used to reduce the data dimension and delete redundant features. Then, in the adaptive group sampling stage, by analyzing the local density information of minority class instances in a low-dimensional state, all minority class instances are divided into different groups according to their respective distribution characteristics, and for minority class instances in different groups, targeted oversampling strategies are used for adaptive oversampling. Finally, the number of samples of the two types in the data set is balanced, solving the imbalance problem from the data level.

[0006] The method of the present invention specifically comprises the following steps:

[0007] Step 1: Preprocess the original data in the software defect dataset, normalize and reduce the dimension of the original data;

[0008] Step 2: Use the data processed in step 1 to train the VAE model. The specific operations are as follows:

[0009] The defective category instance set in the training set processed in step 1 is used as the input data vector of the VAE model. During the training process, the weights of the nodes are updated by minimizing the loss function of the VAE through the back-propagation mechanism; the trained VAE model saves the distribution information of defective category samples, and reconstructs new samples that are similar to but different from the original samples based on the defective category samples input later.

[0010] Step 3: Adaptive group sampling. The specific operations are as follows:

[0011] Step 3-1: Group defective category instances;

[0012] The defective class is set as the minority class instance, and the defect-free class is set as the majority class instance. According to the local density distribution of the minority class instances in the data set, a minority class local density information grouping method MGLDI is proposed.

[0013] According to the local density information of minority class instances, the minority class instance set in the training set after step 1 is divided into four groups: safe group, boundary group, outlier group and dangerous group; the MGLDI method is divided into two steps, the first step is used to determine the minority class instances belonging to the dangerous group and boundary group, and the second step is used to determine the minority class instances belonging to the safe group and outlier group. In the first step, the density expansion idea in the DBSCAN density clustering algorithm is referenced, and three concepts in the DBSCAN density clustering algorithm are redefined: the core object is defined as the majority class object whose majority class instances in the neighborhood exceed the neighborhood density threshold; the boundary object is defined as the majority class object whose density is reachable with the core object, but the majority class instances in the neighborhood are lower than the neighborhood density threshold; the noise object is defined as other majority class instances except the core object and the boundary object;

[0014] In the first step, the training set is first divided into a set of majority class instances and a set of minority class instances. Each minority class instance records two counting marks: a core mark and a boundary mark. The initial values ​​of both marks are zero. Subsequently, each iteration randomly selects an unvisited majority class instance and counts the information of other instances in the neighborhood of the instance. If the instance is a core object, the number of core marks of the minority class instances in its neighborhood is increased by 1. At the same time, the number of boundary marks of the minority class instances in the neighborhood of the boundary object in its neighborhood is increased by 1. Then, other core objects in its neighborhood are added to the queue for the next iteration. If the instance is a noise object, it is ignored. When all majority class instances have been visited, the algorithm stops. Finally, all the marked minority class instances are counted. If the number of core marks of the instance exceeds the specified threshold, the instance is added to the danger group. Otherwise, it is added to the boundary group.

[0015] In the second step, we first exclude the minority class instances that have been marked in the first step, and use the DBSCAN algorithm to perform density clustering on the remaining minority class instances in the training set; after clustering, we add the minority class instances that can be clustered into a security group. For the minority class instances that cannot be clustered into a cluster, we add them to the outlier group.

[0016] Step 3-2: Group oversampling: Use appropriate oversampling methods for minority class instances in different groups to improve the quality of synthetic samples. The specific operations are as follows:

[0017] Step 3-2-1: Develop targeted adaptive oversampling strategies for different groups.

[0018] For the instances in the safety group, they are relatively far from the decision boundary, and compared with the distribution of other groups, the instances in the safety zone are usually more compactly distributed. Therefore, if oversampling is performed directly in the safety zone, it may lead to the creation of meaningless synthetic instances, and the problem of edge distribution may easily occur. Therefore, the present invention uses the SMOTE-Out algorithm as a sampling strategy for the safety zone to expand the distribution range of synthetic instances, reduce the probability of creating meaningless instances, and alleviate the phenomenon of edge distribution to a certain extent.

[0019] For instances in the boundary group, which are near the decision boundary of the two classes, a relatively cautious strategy needs to be adopted when oversampling near the boundary to keep the newly synthesized instances as far away from the overlapping area of ​​the two classes as possible, otherwise it is easy to cause fuzzy class boundary problems. Therefore, the SMOTE algorithm is used with a smaller K value (k=3), and a safety factor is set for the root sample and the auxiliary sample respectively according to the local density information. The more the majority class number around the sample, the lower the safety factor. Let the newly synthesized instances be closer to the samples with high safety factors, thereby reducing the risk of the newly generated instances being in the overlapping area of ​​the boundary.

[0020] Instances out of the group are usually far away from other instances. If the traditional SMOTE method is used to find K nearest neighbors for linear interpolation synthesis, the nearest neighbors found will usually be far away from the original samples and lose their reference significance. In addition, in this case, the new samples will destroy the distribution trend of the original data set and may generate new noise instances. Therefore, VAE is used to reconstruct the instances in the outlier area.

[0021] In the field of software defects, defective instance samples are very valuable resources. In the classification process, it is better to identify non-defective samples as defective samples than to judge defective samples as non-defective samples. Therefore, for minority class instances in the danger group, although they may be noise, this paper chooses to treat them as normal sample points for processing. The sampling strategy adopted is to use the ROS oversampling method to directly copy the original sample as a synthetic sample, in order to prevent the synthesis of new noise points around the original instance through the VAE method or linear interpolation method.

[0022] Step 3-2-2: Each time, randomly select a minority class instance from the training set processed in step 1. If the instance belongs to the safe group, use the SMOTE-Out algorithm to synthesize a new instance for the instance; if the instance belongs to the border group, use the SL-SMOTE algorithm with a smaller k-nearest neighbor parameter (k=3) to synthesize a new instance for the instance; if the instance belongs to the outlier group, input the instance into the trained VAE model and use VAE to reconstruct a new instance for the instance; if the instance belongs to the dangerous group, use the ROS algorithm to synthesize a new instance for the instance. Finally, add the synthesized new instance to the training set.

[0023] Step 3-3: Repeat step 3-2-2 until the number of instances of the two categories in the training set is equal and the number of data of the two categories in the training set is balanced.

[0024] Preferably, the raw data in the software defect data set is preprocessed to normalize and reduce the dimension of the raw data; specifically, the following steps are performed:

[0025] Step 1-1: Divide the original data into training set and test set in proportion. Use the Min-Max normalization method to perform linear transformation on the training set, map the eigenvalues ​​to [0,1], and delete duplicate and redundant instances after processing; Min-Max normalization method formula (1):

[0026]

[0027] Where x represents the original eigenvalue; x min represents the minimum value of the eigenvalue in the training set; x max represents the maximum value of the eigenvalue in the training set; x * represents the normalized eigenvalue.

[0028] Step 1-2: Assume that the training set processed in step 1-1 is represented as D, where the defective class is the minority class, represented as D+, and the non-defective class is the majority class, represented as D-; first, perform Agglomerative-Clustering hierarchical clustering on D-. The number of clusters is set to the result of dividing the number of majority class instances in D by the number of minority class instances and rounding. Initially, each sample in D- is regarded as a cluster, and then repeated iterations begin. In each iteration, the two clusters with the smallest specified distance are merged until the number of clusters meets the conditions. The calculation formula for the distance between two clusters is the average distance between the sample pairs in the two clusters. Subsequently, the cluster center sample is selected from each cluster and merged with the original minority class set to obtain a balanced data set D′. The cluster center sample is defined as the sample with the smallest average distance to other instances in each cluster. Finally, the relief algorithm is used to calculate the feature weights on D′, the feature weights are sorted in descending order, and the top k features are retained according to the sorting results.

[0029] Assume there are clusters G and H, n G Indicates the number of samples in cluster G, n H represents the number of samples in cluster H, then the distance between clusters d avg Calculated by formula (2). The distance d between samples is specified as the Euclidean distance;

[0030]

[0031] Beneficial results of the present invention:

[0032] 1. This technology adds a class imbalance processing strategy to the classic Relief feature selection method, so that the improved Relief feature selection method can select a representative feature subset on a class imbalanced data set, improve the quality of training features, and reduce computational complexity.

[0033] 2. This technology fully considers the different characteristics of the distribution of minority class samples in the imbalanced data set, and adopts targeted oversampling strategies according to the minority class samples with different characteristics, thereby improving the quality of the synthesized minority class instances and the accuracy of the classifier in classifying minority class instances. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1 The overall flow chart of software defect prediction technology based on minority class instance oversampling algorithm;

[0035] Figure 2 Description of the AgglomerativeClustering-Relief algorithm;

[0036] Figure 3 VAE structure diagram;

[0037] Figure 4 MGLDI algorithm description;

[0038] Figure 5 Diagram of security group instance oversampling strategy;

[0039] Figure 6 Schematic diagram of oversampling strategy for boundary group instances;

[0040] Figure 7 Schematic diagram of the out-of-group instance oversampling strategy. DETAILED DESCRIPTION

[0041] The present invention is described in detail below with reference to the accompanying drawings and in combination with the software defect prediction data set. Figure 1 As shown, the specific steps are as follows:

[0042] Step 1: Use the Min-Max normalization method to process the original training set, map the feature values ​​to between [0, 1], and delete the duplicate and redundant instances after the processing is completed. After the training set is normalized, use the AgglomerativeClustering-Relief feature selection algorithm proposed in the present invention to reduce the data dimension and delete redundant features. The AgglomerativeClustering-Relief algorithm is described as follows Figure 2 shown.

[0043] Step 2: On the training set processed in step 1, use all minority class instances as the input vectors of the VAE model to train the model. The VAE model structure is shown in the figure below: Figure 3 shown.

[0044] Step 3: On the training set processed in step 1, the MGLDI grouping method proposed in the present invention is used to group all minority class instances according to the local density information. The minority class instances are divided into four groups: safe group, border group, outlier group, and dangerous group. The MGLDI algorithm is described as follows: Figure 4 shown.

[0045] Step 4: Randomly select a minority class instance from the training set processed in step 1. If the instance belongs to the security group, use the SMOTE-Out algorithm to synthesize a new instance from the instance. The schematic diagram of the security group oversampling strategy is as follows: Figure 5 As shown in the figure, the solid triangles represent the distribution of root samples. The dotted triangles represent the distribution of direct linear interpolation synthetic instances. The solid five-pointed stars represent the distribution of SMOTE-Out synthetic instances. The other hollow triangles and hollow circles represent other minority class instances and other majority class instances respectively; if the instance belongs to the boundary group, the SL-SMOTE algorithm is used to synthesize a new instance for the instance. The schematic diagram of the boundary group oversampling strategy is shown in Figure 6 As shown in the figure, the solid triangles represent the distribution of root samples. The dotted triangles represent the distribution of direct linear interpolation synthetic instances. The solid five-pointed stars represent the distribution of SL-SMOTE synthetic instances. The other hollow triangles and hollow circles represent other minority class instances and other majority class instances respectively; if the instance belongs to the outlier group, the VAE reconstruction method is used to synthesize a new instance for the instance. The schematic diagram of the outlier group oversampling strategy is shown in Figure 7 As shown in the figure, the solid triangles represent the distribution of root samples. The dotted triangles represent the distribution of direct linear interpolation synthetic instances. The solid five-pointed stars represent the distribution of VAE synthetic instances. The other hollow triangles and hollow circles represent other minority class instances and other majority class instances respectively; if the instance belongs to the dangerous group, the ROS algorithm is used to synthesize a new instance for the instance. Finally, the synthesized new instance is added to the training set.

[0046] Step 5: Repeat step 4 until the number of instances of the two categories in the training set is equal.

Claims

1. A method for adaptive oversampling of software defect data based on local density information, characterized in that: The method specifically comprises the following steps: Step 1: Preprocess the original data in the software defect dataset, normalize and reduce the dimension of the original data; Step 2: Use the data processed in step 1 to train the VAE model; Step 3: Adaptive group sampling. The specific operations are as follows: Step 3-1: Group defective category instances; The defective class is set as the minority class instance, and the defect-free class is set as the majority class instance. According to the local density distribution of the minority class instances in the data set, a minority class local density information grouping method MGLDI is proposed. According to the local density information of minority class instances, the minority class instance set in the training set after step 1 is divided into four groups: safe group, boundary group, outlier group and dangerous group; the MGLDI method is divided into two steps. The first step is used to determine the minority class instances belonging to the dangerous group and the boundary group, and the second step is used to determine the minority class instances belonging to the safe group and the outlier group; the first step refers to the density expansion idea in the DBSCAN density clustering algorithm, and redefines three concepts in the DBSCAN density clustering algorithm: define the core object as the majority class object whose majority class instances in the neighborhood exceed the neighborhood density threshold; define the boundary object as the majority class object whose density is reachable with the core object, but whose majority class instances in the neighborhood are lower than the neighborhood density threshold; define the noise object as the majority class instances other than the core object and the boundary object; In the first step, the training set is first divided into a set of majority class instances and a set of minority class instances. Each minority class instance records two counting marks: a core mark and a boundary mark, and the initial values ​​of both marks are zero; then, each iteration randomly selects an unvisited majority class instance and counts the information of other instances in the neighborhood of the instance; if the instance is a core object, the number of core marks of the minority class instances in its neighborhood is increased by 1, and at the same time, the number of boundary marks of the minority class instances in the neighborhood of the boundary object in its neighborhood is increased by 1; then other core objects in its neighborhood are added to the queue for the next iteration; if the instance is a noise object, it is ignored; when all majority class instances have been visited, the algorithm stops; finally, all marked minority class instances are counted, and if the number of core marks of the instance exceeds the specified threshold, the instance is added to the danger group; otherwise, it is added to the boundary group; In the second step, we first exclude the minority class instances that have been marked in the first step, and use the DBSCAN algorithm to perform density clustering on the remaining minority class instances in the training set. After clustering, the minority class instances that can be clustered are added to the security group. For the minority class instances that cannot be clustered, they are added to the outlier group. Step 3-2: Group oversampling: Use appropriate oversampling methods for minority class instances in different groups to improve the quality of synthetic samples. The specific operations are as follows: Step 3-2-1: Develop targeted adaptive oversampling strategies for different groups; For instances in the security group, the SMOTE-Out algorithm is used as the sampling strategy for the security zone to expand the distribution range of synthetic instances, reduce the probability of creating meaningless instances, and alleviate the phenomenon of edge distribution to a certain extent; For instances in the boundary group, the SMOTE algorithm is used with a smaller K value, and a safety factor is set for the root sample and the auxiliary sample respectively according to the local density information; the more the majority class number around the sample, the lower the safety factor; the newly synthesized instance is made closer to the sample with a high safety factor, thereby reducing the risk of the newly generated instance being in the boundary overlap area; For the outlier instances, VAE is used to reconstruct them; For minority class instances in the risk group, the ROS oversampling method is used to directly copy the original samples as synthetic samples to prevent new noise points from being synthesized around the original instances by the VAE method or linear interpolation method; Step 3-2-2: Each time, randomly select a minority class instance from the training set processed in step 1; if the instance belongs to the safe group, use the SMOTE-Out algorithm to synthesize a new instance for the instance; if the instance belongs to the border group, use the SL-SMOTE algorithm, set the k-nearest neighbor parameter to 3, and synthesize a new instance for the instance; if the instance belongs to the outlier group, input the instance into the trained VAE model, and use VAE to reconstruct a new instance for the instance; if the instance belongs to the dangerous group, use the ROS algorithm to synthesize a new instance for the instance; finally, add the synthesized new instance to the training set; Step 3-3: Repeat step 3-2-2 until the number of instances of the two categories in the training set is equal and the number of data of the two categories in the training set is balanced.

2. The method for adaptive oversampling of software defect data based on local density information according to claim 1, characterized in that: The raw data in the software defect data set is preprocessed to normalize and reduce the dimension of the raw data; specifically: Step 1-1: Divide the original data into training set and test set in proportion; use the Min-Max normalization method to perform linear transformation on the training set, map the eigenvalues ​​to [0,1], and delete duplicate and redundant instances after processing; Min-Max normalization method formula (1): Where x represents the original eigenvalue; x min represents the minimum value of the eigenvalue in the training set; x max represents the maximum value of the eigenvalue in the training set; x * represents the normalized eigenvalue; Step 1-2: Assume that the training set processed in step 1-1 is represented as D, where the defective class is the minority class, represented as D+, and the non-defective class is the majority class, represented as D-; first, perform Agglomerative-Clustering hierarchical clustering on D-; the number of clusters is the result of dividing the number of majority class instances by the number of minority class instances in D and rounding; initially, each sample in D- is regarded as a cluster, and then repeated iterations are started; in each iteration, the two clusters with the smallest specified distance are merged until the number of clusters meets the condition; The distance between two clusters is calculated as the average of the distances between pairs of samples in the two clusters; Then, the cluster center samples are selected from each cluster and merged with the original minority class set to obtain a balanced data set D′; the cluster center samples are defined as the samples with the smallest average distance to other instances in each cluster; finally, the feature weights are calculated on D′ using the Relief algorithm, the feature weights are sorted in descending order, and the top k features are retained according to the sorting results; Assume there are clusters G and H, n G Indicates the number of samples in cluster G, n H Represents the number of samples in cluster H, then the distance between clusters d avg Calculated by formula (2); where the distance d between samples is specified as the Euclidean distance; 3. The method for adaptive oversampling of software defect data based on local density information according to claim 1, characterized in that: The VAE model is trained as described above; the specific operations are as follows: The set of defective category instances in the training set processed in step 1 is used as the input data vector of the VAE model; during the training process, the weights of the nodes are updated by minimizing the loss function of the VAE through the back-propagation mechanism; the trained VAE model saves the distribution information of the defective category samples, and reconstructs new samples that are similar to but different from the original samples based on the defective category samples input subsequently.

Images