A software defect data generation method and system

Abstract

The application relates to a software defect data generation method and system, which comprises the following steps: S1, obtaining a software data set, wherein the software data set comprises a defect data set and a non-defect data set, the defect data set is taken as a minority class data set, and the non-defect data set is taken as a majority class data set; S2, oversampling the defect data set and the non-defect data set to generate new defect samples, and putting the new defect samples into the defect data set so that sample points of the defect data set and sample points of the non-defect data set reach balance after the new defect samples are put in. The application can balance the defect data set and the non-defect data set in the software data set. Experiments prove that the new defect data set constructed by the application can enable a model used for software defect prediction to learn information with distinguishability, and ensure the high efficiency of the model.

Description

Technical Field

[0001] This invention relates to the field of data generation technology, and in particular to a method and system for generating software defect data. Background Technology

[0002] Software Defect Prediction (SDP) involves predicting potential defects in software. Effective defect prediction helps identify defective areas in software in a timely manner. In the early stages of the software development lifecycle, identifying and eliminating software defects is crucial for producing cost-effective and high-quality software products. Typically, SDP datasets exhibit imbalance, meaning that the number of non-defect (majority class) samples is large, while the number of defective (minority class) samples is small. Training models on imbalanced data leads to biased learning, resulting in inaccurate predictions. Therefore, effectively handling imbalanced data is essential for successfully developing effective defect prediction models.

[0003] Oversampling methods address imbalanced data distribution by increasing the number of minority class samples. SMOTE (Synthetic Minority Oversampling Technique) is the most classic oversampling method, which increases the number of minority class samples by randomly selecting minority class samples and performing linear interpolation. However, SMOTE ignores the impact of noisy or useless samples on the classification results, potentially generating new noisy or useless samples and leading to overfitting. Cluster-SMOTE (Cluster Synthetic Minority Oversampling Technique) is an improvement on SMOTE. By clustering minority class samples, it better preserves the local structure of the data and reduces the risk of overfitting. However, this method may also generate new noisy points, making it difficult for the classifier to learn discriminative information. Summary of the Invention

[0004] Therefore, the technical problem to be solved by the present invention is to overcome the imbalance between defect data and non-defect data in the software defect prediction data in the prior art, and the noise in the new data constructed by the existing methods.

[0005] To address the aforementioned technical problems, this invention provides a method for generating software defect data, comprising:

[0006] Step S1: Obtain a software dataset, wherein the software dataset includes a defect dataset and a non-defect dataset, with the defect dataset being the minority class dataset and the non-defect dataset being the majority class dataset;

[0007] Step S2: Oversample the defect dataset and the non-defect dataset to generate the required new defect samples. Put the new defect samples into the defect dataset so that the sample points of the defect dataset and the non-defect dataset are balanced after the new defect samples are put in.

[0008] In one embodiment of the present invention, step S1 further includes: a method for calculating the number of new defect samples to be generated based on the acquired software dataset, specifically:

[0009] Obtain the software dataset D = {(x i y i )|x i ∈R d y i ∈{0, 1}, i = 1, ..., n}, where n represents the number of software data, d represents the number of features of the software data, and x i Let y represent the i-th piece of software data. i It is x i The tag, if y i =0, then x i This is non-defect data; if y i =1, then x i This is defect data;

[0010] Let D = D min ∪D maj , where D maj For the majority class dataset, the number of samples is denoted as n. maj ;D min This is a minority class dataset, with its sample size denoted as n. min And n maj >n min ;

[0011] Let n be the number of new defect samples that need to be generated. new =n maj -n min .

[0012] In one embodiment of the present invention, step S2 involves oversampling the defective dataset and the non-defective dataset to generate the required new number of defective samples. The method includes:

[0013] For the minority class dataset D min Preprocessing is performed to obtain a safe sample. Dangerous samples and the minority class dataset D′ after removing noisy samples min ;

[0014] For the majority class dataset D majPreprocessing is performed to obtain the majority class dataset D′ after noise removal. maj , and based on D′ maj Generate a non-defect boundary data set B maj ;

[0015] According to the non-defect boundary data set B maj Calculate the security sample The probability of selecting a new defect sample point as the desired new sample point is determined from the selected points.

[0016] For the dangerous sample Clustering generates sub-clusters, and weights W are assigned to these sub-clusters. c ;

[0017] According to the security sample Quantity, hazardous samples Quantity, non-defect boundary data set B maj Selection probability and weight W c The number of new defect samples required to generate.

[0018] In one embodiment of the present invention, the minority class dataset D is... min Preprocessing is performed to obtain a safe sample. Dangerous samples and the minority class dataset D′ after removing noisy samples min The methods include:

[0019] Find each sample x in the software dataset D i ∈D min The k nearest neighbor samples, where k is a preset parameter;

[0020] For each sample x i ∈D min Count the number m1 of the majority class samples among its k nearest neighbors, and set a threshold k1. If k = m1, then x i It is considered a noise sample; if m1 < k1, then x i Considered a safe sample, otherwise x i Considered a dangerous sample;

[0021] The set consisting of all safe samples is denoted as The number of its sample points is denoted as The set consisting of all hazardous samples is denoted as The number of its sample points is recorded as From x i The set of all noise samples is denoted as

[0022] make For the minority class dataset after removing noisy samples, and remember D′ min The number of samples in the middle.

[0023] In one embodiment of the present invention, the majority class dataset D is... maj Preprocessing is performed to obtain the majority class dataset D′ after noise removal. maj , and based on D′ maj Generate a non-defect boundary data set B maj The methods include:

[0024] Find each sample x in the software dataset D j ∈D maj Given k nearest neighbor samples, count the number m2 of minority class samples among the k nearest neighbor samples;

[0025] For each sample x j ∈D maj If m2 = k, then x j It is considered a noise sample;

[0026] From x j The set of all noise samples is denoted as The number of its sample points is denoted as make The majority class dataset after removing noisy samples, and its number of sample points is...

[0027] In data set D′ maj Search for each sample Find the k2 nearest neighbor samples and put them all into the non-defect boundary data set B. maj middle.

[0028] In one embodiment of the present invention, the step of using the non-defect boundary data set B... maj Calculate the security sample The selection probability of a point is chosen as the new defect sample point to be generated, and the method includes:

[0029] For each sample Calculate its selection probability SP(x) i The formula is:

[0030]

[0031] Among them, Sw(x) i ) represents sample x i The relative importance is determined by the formula:

[0032] Sw(x i )=F(x i )×G(x i )

[0033] F(x i ) represents sample x i With non-defect boundary dataset B maj The formula for the degree of closeness is:

[0034]

[0035] Where, f(x) i x j ) represents sample x i With x j ∈B maj The formula for the degree of closeness is:

[0036]

[0037] d(x i x j ) represents sample x i With x j The Euclidean distance between them;

[0038] G(x i ) represents sample x i Compared to the non-defect boundary dataset B maj The density is given by the formula:

[0039]

[0040] g(x i x j ) indicates a sample With x j ∈B maj The distance is given by the formula:

[0041]

[0042] Where M is a user-defined parameter.

[0043] In one embodiment of the present invention, the dangerous sample Clustering generates sub-clusters, and weights W are assigned to these sub-clusters. c The methods include:

[0044] dangerous samples Perform k-means clustering to generate nc sub-clusters;

[0045] Calculate the risk level of each cluster sub-cluster in the sample The relative weights in the formula are:

[0046]

[0047] in, This represents the number of samples in the c-th sub-cluster.

[0048] In one embodiment of the invention, the step of basing the security sample Quantity, hazardous samples Quantity, non-defect boundary data set B maj Selection probability and weight W c Methods for generating the required number of new defect samples include:

[0049] According to the security sample and dangerous samples The number of safe samples and dangerous samples The number of new samples to be generated respectively and The formula is:

[0050]

[0051]

[0052] Based on each cluster sub-cluster in The relative weight W in c Calculate the number of new samples needed to generate for each cluster sub-cluster. in, The formula represents the number of new samples needed to generate for the c-th sub-cluster.

[0053]

[0054] For safe samples New sample generation: First, based on the selection probability In the set Select a minority class safe sample, denoted as x. i Then from the non-defect boundary data set B maj Let x be a randomly selected majority class boundary sample. j The feature interpolation for these two samples is performed using the following formula:

[0055]

[0056] in, This represents the k-th feature value of a newly generated sample. The interpolation coefficients α∈(0,1) are randomly generated. This step is repeated until the number of newly generated samples at the safe sample points reaches a certain threshold. Make safe samples The newly generated sample set is

[0057] For dangerous samples New sample generation: for defect hazard datasets The c-th cluster is processed as follows: two sample points x are randomly selected from this cluster. i x j According to the formula Perform feature interpolation; repeat this process until the number of newly generated samples in the sub-cluster reaches a certain threshold. Dangerous samples The newly generated sample set is

[0058] Finally, a newly generated set of defect data samples is obtained.

[0059] In one embodiment of the present invention, if Therefore, regardless of the value of nc, only one cluster can be generated.

[0060] To address the aforementioned technical problems, this invention provides a software defect data generation system, comprising:

[0061] Acquisition module: used to acquire software datasets, wherein the software datasets include defect datasets and non-defect datasets, with the defect datasets being treated as minority class datasets and the non-defect datasets being treated as majority class datasets;

[0062] Oversampling module: used to oversample the defect dataset and non-defect dataset to generate new defect samples as needed, and put the new defect samples into the defect dataset so that the sample points of the defect dataset and non-defect dataset are balanced after the new defect samples are added.

[0063] The technical solution of the present invention has the following advantages compared with the prior art:

[0064] This invention constructs a new defect dataset that balances the defect dataset (minority class dataset) and the non-defect dataset (majority class dataset) in software datasets, and removes the influence of noise, thus ensuring the effectiveness of subsequent model training and prediction.

[0065] The novel defect dataset constructed in this invention enables models used for software defect prediction to learn discriminative information, ensuring the efficiency of the models. Attached Figure Description

[0066] To make the content of this invention easier to understand, the invention will be further described in detail below with reference to specific embodiments and accompanying drawings.

[0067] Figure 1 This is a flowchart of the method of the present invention. Detailed Implementation

[0068] The present invention will be further described below with reference to the accompanying drawings and specific embodiments, so that those skilled in the art can better understand and implement the present invention. However, the embodiments described are not intended to limit the present invention.

[0069] Example 1

[0070] Reference Figure 1 As shown, this invention relates to a method for generating software defect data, comprising:

[0071] Step S1: Obtain a software dataset, wherein the software dataset includes a defect dataset and a non-defect dataset, with the defect dataset being the minority class dataset and the non-defect dataset being the majority class dataset;

[0072] Step S2: Oversample the defect dataset and the non-defect dataset to generate the required new defect samples. Put the new defect samples into the defect dataset so that the sample points of the defect dataset and the non-defect dataset are balanced after the new defect samples are put in.

[0073] The following is a detailed description of the present invention:

[0074] I. Data Input

[0075] Input software dataset D = {(x i y i )|x i ∈R d y i ∈{0, 1}, i = 1, ..., n}, where n represents the number of software data, d is the number of features of the software data, and x i Let y represent the i-th piece of software data. i It is its label, if y i =0, then x i This is non-defect data; if y i =1, then x i This is defect data.

[0076] Let D = D min ∪D maj D maj The majority class dataset, i.e., the dataset of non-defective items, has a sample size denoted as n. maj ;D minThis is a minority class dataset, specifically a sample dataset of defective projects, with the number of samples denoted as n. min Let n be the number of newly generated sample points. new =n maj -n min .

[0077] II. Oversampling

[0078] Let the newly generated defective project sample data set be D. new Initialize it as an empty set. The oversampling steps are as follows:

[0079] 1. Defect dataset D min Preprocessing of (i.e., minority class datasets)

[0080] (1) Find each sample x in the dataset D. i ∈D min The k nearest neighbor samples, where k is a pre-set parameter;

[0081] (2) Count the number m1 of the majority class samples among these k nearest neighbors, and set a threshold k1. If k = m1, then x i These are considered noise sample points; if m1 < k1, then x i Considered a safe sample, otherwise x i It is considered a dangerous sample.

[0082] (3) The set consisting of all safe samples is denoted as The number of its sample points is denoted as The set consisting of all hazardous sample data is denoted as The number of its sample points is recorded as The set consisting of all noise samples is denoted as

[0083] (4) Let The minority class dataset after removing noisy samples, i.e. And remember D′ min The number of samples in the middle.

[0084] 2. Non-defect dataset D maj Preprocessing of (majority class dataset)

[0085] (1) Find each sample x in the dataset D. j ∈D maj Given k nearest neighbor samples, count the number m2 of minority class samples among these k nearest neighbor samples;

[0086] (2) If m2 = k, then x j These are considered noise sample points.

[0087] (3) The set consisting of all noise samples is denoted as The number of its sample points is denoted as make Let be the majority class dataset after removing noisy samples, and its number of sample points be . Let the denoised dataset be D′=D′ min ∪D′ maj .

[0088] (4) In the data set D′ maj Search for each sample Find the k2 nearest neighbors and put them into the non-defect boundary dataset B. maj middle.

[0089] 3. Calculate the safe samples in the defect data. Selection probability

[0090] For each Calculate its selection probability SP(x) i The formula is as follows:

[0091]

[0092] Where Sw(x) i ) represents sample x i The relative importance, that is

[0093] Sw(x i )=F(x i )×G(x i )

[0094] F(x i ) represents sample x i With non-defect boundary dataset B maj The degree of closeness, that is

[0095]

[0096] Where f(x) i x j ) represents sample x i With x j (x j ∈B maj The degree of closeness of ) is defined as:

[0097]

[0098] d(x i x j ) represents sample x i With x j The Euclidean distance between them. G(x)i ) represents sample x i Compared to the non-defect boundary dataset B maj The density, that is

[0099]

[0100] g(x i x j ) represents sample x i With x j (x j ∈B maj The distance, i.e.

[0101]

[0102] M is a user-defined parameter.

[0103] 4. For hazardous samples in the defect data. Cluster and assign weights to the sub-clusters.

[0104] (1) Sample the defect risk dataset Perform k-means clustering to generate n c There are n clusters, where n are subclusters. c These are user-defined parameters. It's important to note that if... Then regardless of n c Regardless of the value, only one cluster can be generated.

[0105] (2) Calculate the clustering subclusters in The relative weights in, i.e.

[0106]

[0107] in This represents the number of samples in the c-th sub-cluster.

[0108] 5. Generate new minority class samples (i.e., the required number of new defective samples).

[0109] (1) Based on the number of safe samples and hazardous samples, calculate the number of new samples that need to be generated for each type of sample. and Right now

[0110]

[0111]

[0112] Based on each cluster sub-cluster in The relative weights in the formula are used to calculate the number of new samples needed to generate for each cluster sub-cluster. in This indicates the number of new samples needed to be generated for the c-th sub-cluster.

[0113]

[0114] (2) For the generation of new samples for safe sample points: first, based on the selection probability. In the set Select a minority class safe sample, denoted as x. i Then from the non-defect boundary data set B maj Let x be a randomly selected majority class boundary sample. j ; Perform feature interpolation on these two samples, that is:

[0115]

[0116] in This represents the k-th feature value used to generate a new sample, where the interpolation coefficient α∈(0,1) is randomly generated. This step is repeated until the number of newly generated samples at the safe sample points reaches a certain threshold. Let the newly generated sample set of safe sample points be

[0117] (3) New sample generation for hazardous sample points: For the defect hazard data set The c-th (c = 1, ..., n) c Perform the following operation on each cluster sub-cluster: randomly select two sample points x from each cluster sub-cluster. i x j According to the formula Perform feature interpolation; repeat this process until the number of newly generated samples in the sub-cluster reaches a certain threshold. Let the newly generated sample set of dangerous sample points be

[0118] (4) Finally, the newly generated defect data sample set is obtained.

[0119] III. Model Training and Prediction

[0120] Let D * =D new ∪D is the total set of software project data samples after oversampling. D′ is used as the input data for training the model, and a mature classification technique is used to train a predictive model for software defect assessment.

[0121] The present invention is further illustrated by a specific embodiment below:

[0122] This invention was tested on the Tomcat 6.0 dataset in the PROMISE repository. The dataset consists of 20 software metrics and a dependent variable indicating the number of defects in a specific category. In software engineering, various metrics are commonly used to measure characteristics such as software quality, complexity, and maintainability. Important metrics in Tomcat 6.0 include CBO, RFC, LCOM, LOC, MOA, AMC, Max_CC, and Avg_CC. For example, CBO (Coupling Between Objects) represents the coupling between objects, RFC (Response For a Class) represents the number of methods a class responds to, LCOM (Lack of Cohesion of Methods) represents the lack of cohesion in methods, LOC (Lines Of Code) represents the number of lines of code, MOA (Number Of Attributes) represents the number of attributes in a class, and AMC (Average Method Complexity) represents the average method complexity, etc. Software projects are classified into those with no defects (the majority class) or those with defects (the minority class) based on attribute descriptions. The specific implementation steps are as follows:

[0123] I. Data Input

[0124] Input software dataset D = {(x i y i )|x i ∈R d y i ∈{0, 1}, i = 1, ..., n}, where n represents the number of software data, d is the number of features of the software data, and x i Let y represent the i-th piece of software data. i It is its label. If y i =0, then x i This is non-defect data; if y i =1, then x i This is defect data. In this example, n = 352, d = 20.

[0125] Let D = D min ∪D maj D maj The majority class dataset, i.e., the dataset of non-defective items, has a sample size denoted as n. maj ;D min This is a minority class dataset, specifically a sample dataset of defective projects, with the number of samples denoted as n. min Let n be the number of newly generated sample points. new =n maj -n minThe goal is to balance the number of defective and non-defective data. In this example, n maj =312, n min =40, n new =272.

[0126] II. Oversampling

[0127] Let the newly generated defective project sample data set be , and initialize it as an empty set. The oversampling steps are as follows:

[0128] 1. Defect dataset D min Preprocessing of (i.e., minority class datasets)

[0129] (1) Find each sample x in the dataset D. i ∈D min The k nearest neighbor samples are selected, where k is a pre-defined parameter; in this example, k = 5.

[0130] (2) Count the number m1 of the majority class samples among these k nearest neighbors, and set a threshold k1. If k = m1, then x i These are considered noise sample points; if m1 < k1, then x i Considered a safe sample, otherwise x i This is considered a dangerous sample. In this example, k = 5, k1 = 3.

[0131] (3) The set consisting of all safe samples is denoted as The number of its sample points is denoted as The set consisting of all hazardous sample data is denoted as The number of its sample points is recorded as The set consisting of all noise samples is denoted as

[0132] (4) Let The minority class dataset after removing noisy samples, i.e. And remember D′ min The number of samples in the middle. In this example, n′ min =40.

[0133] 2. Non-defect dataset D maj Preprocessing of (majority class dataset)

[0134] (1) Find each sample x in the dataset D. j ∈D maj Given k nearest neighbor samples, count the number of minority class samples m2 among these k nearest neighbor samples;

[0135] (2) If k = m2, then xj These are considered noise sample points.

[0136] (3) The set consisting of all noise samples is denoted as The number of its sample points is denoted as make Let be the majority class dataset after removing noisy samples, and its number of sample points be . Let the denoised dataset be D′=D′ min ∪D′ maj .

[0137] (4) In the data set D′ maj Search for each sample Find the k2 nearest neighbor samples and put them into the non-defect boundary data set B. maj In this example, k2 = 3.

[0138] 3. Calculate the safe samples in the defect data. Selection probability

[0139] For each Calculate its selection probability SP(x) i The formula is as follows:

[0140]

[0141] Where Sw(x) i ) represents sample x i The relative importance, that is

[0142] Sw(x i )=F(x i )×G(x i )

[0143] F(x i ) represents sample x i With non-defect boundary dataset B maj The degree of closeness, that is

[0144]

[0145] Where f(x) i x j ) represents sample x i With x j (x j ∈B maj The degree of closeness of ) is defined as:

[0146]

[0147] d(x i x j) represents sample x i With x j The Euclidean distance between them. G(x) i ) represents sample x i Compared to the non-defect boundary dataset B maj The density, that is

[0148]

[0149] g(x i x j ) represents sample x i With x j (x j ∈B maj The distance, i.e.

[0150]

[0151] M is a user-defined parameter. In this example, M = 0.2.

[0152] DF(x i ) represents sample x i Relative to sample set B maj Density, i.e.

[0153]

[0154] 4. For hazardous samples in the defect data. Cluster and assign weights to the sub-clusters.

[0155] (1) Sample the defect risk dataset Perform k-means clustering to generate n c There are n clusters, where n are subclusters. c These are user-defined parameters. Note that if... Then regardless of n c Regardless of the value of n, only one sub-cluster can be generated. In this example, n c =5.

[0156] (2) Calculate the clustering subclusters in The relative weights in, i.e.

[0157]

[0158] in This represents the number of samples in the i-th sub-cluster.

[0159] 5. Generate new minority class samples (i.e., the required number of new defective samples).

[0160] (1) Based on the number of safe samples and hazardous samples, calculate the number of new samples that need to be generated for each type of sample. and

[0161]

[0162]

[0163] In this example, Based on each cluster sub-cluster in The relative weights in the formula are used to calculate the number of new samples needed to generate for each cluster sub-cluster. in This indicates the number of new samples needed to be generated for the c-th sub-cluster.

[0164]

[0165] In this example,

[0166] (2) For the generation of new samples for safe sample points: first, based on the selection probability. In the set Select a minority class safe sample, denoted as x. i Then, randomly select a majority class boundary sample from the non-defect boundary data set, denoted as x. j ; Perform feature interpolation on these two samples, that is:

[0167]

[0168] in This represents the k-th feature value used to generate a new sample, where the interpolation coefficient α∈(0,1) is randomly generated. This step is repeated until the number of newly generated samples at the safe sample points reaches a certain threshold. Let the newly generated sample set of safe sample points be

[0169] (3) New sample generation for hazardous sample points: For the defect hazard data set The c-th (c = 1, ..., n) c Perform the following operation on each cluster sub-cluster: randomly select two sample points x from each cluster sub-cluster. i x j Feature interpolation is performed according to formula (1); this process is repeated until the number of newly generated samples in the sub-cluster reaches a certain threshold. Let the newly generated sample set of dangerous sample points be

[0170] (4) Finally, the newly generated defect data sample set is obtained.

[0171] III. Model Training and Prediction

[0172] Let D * =D new ∪D is the total set of software project data samples after oversampling. D′ is used as the input data for training the model, and a mature classification technique is used to train a predictive model for software defect assessment.

[0173] IV. Experimental Comparison

[0174] This embodiment performs a five-fold cross-validation experiment on the Tomcat 6.0 dataset. The received software data sample x... i ∈R d The data is input into a software defect assessment model, and the output is used to determine the quality of the data samples from the unknown software project. The dataset is randomly divided into five parts, one of which serves as the test set, and the other four parts as the training set for generating new data samples. After sample generation, the test set is classified using GaussianNB (Gaussian Naive Bayes) and SVM (Support Vector Machine) to classify the new dataset obtained after oversampling. In this embodiment, the average value obtained from ten five-fold cross-validations is used as the result of the experiment. In the oversampling method, SMOTE and Cluster-SMOTE are used as the comparison methods.

[0175] The classification performance was evaluated using three different evaluation metrics: AUC, G-mean, and Recall. The confusion matrix is ​​shown in Table 1.

[0176] Table 1 Confusion Matrix

[0177] Predicted value = 1 Predicted value = 0 True value = 1 TP FN True value = 0 FP TN

[0178]

[0179]

[0180]

[0181] AUC=Area Under Receiver Operating Characteristic Curve

[0182] The classification results are shown in Table 2. Compared with SMOTE and Cluster-SMOTE, the present invention performs better in terms of AUC, G-mean, and Recall.

[0183] Table 2 Comparison of results from three oversampling methods

[0184]

[0185] Example 2

[0186] This embodiment provides a software defect data generation system, including:

[0187] Acquisition module: used to acquire software datasets, wherein the software datasets include defect datasets and non-defect datasets, with the defect datasets being treated as minority class datasets and the non-defect datasets being treated as majority class datasets;

[0188] Oversampling module: used to oversample the defect dataset and non-defect dataset to generate new defect samples as needed, and put the new defect samples into the defect dataset so that the sample points of the defect dataset and non-defect dataset are balanced after the new defect samples are added.

[0189] Although preferred embodiments of this application have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of this application.

[0190] Obviously, the above embodiments are merely illustrative examples for clear explanation and are not intended to limit the implementation. Those skilled in the art will recognize that other variations or modifications can be made based on the above description. It is neither necessary nor possible to exhaustively list all possible implementations here. However, obvious variations or modifications derived therefrom are still within the scope of protection of this invention.

Claims

1. A method of software defect data generation, the method comprising: The application relates to a method for generating new defect samples in a software defect data set, and a device thereof. The method comprises the following steps: Step S1: acquiring a software data set, wherein the software data set comprises a defect data set and a non-defect data set, the defect data set is taken as a minority class data set, and the non-defect data set is taken as a majority class data set; Obtain software dataset ,in, Indicates the number of software data items. The number of features representing software data. Indicates the first Software data, yes The tag, if ,but This is non-defect data; if ,but This is defect data; Let where, is a majority class dataset with samples and is a minority class dataset with samples and ; Let the number of new defect samples to be generated be represented as ; The step S1 further comprises the following steps of calculating the number of required new defect samples according to the acquired software data set, specifically: Step S2: oversampling the defect data set and the non-defect data set to generate the required new defect samples, and putting the new defect samples into the defect data set so that the sample points of the defect data set and the sample points of the non-defect data set reach balance after the new defect samples are put in; preprocessing the minority class dataset to obtain safe samples , dangerous samples and a minority class dataset after removing noise samples ; For the majority class dataset Preprocessing is performed to obtain the majority class dataset after noise removal. and based on Generate a set of non-defect boundary data ; According to the non-defect boundary data set , calculate the selection probability of the safety sample selected as the new defect sample point required to be generated; to the dangerous sample The cluster generates cluster sub-clusters and assigns weights to the cluster sub-clusters ; According to the number of safety samples , the number of dangerous samples , the non-defect boundary data set , the selection probability and the weight to generate the required new defect sample number.

2. The software defect data generation method of claim 1, wherein: The method for processing the minority class dataset Preprocessing to obtain safe samples , dangerous samples And the minority class dataset after removing noise samples , the method comprises: In the software dataset for each sample a number of nearest neighbor samples, is a preset parameter; For each sample , count the number of majority class samples among its nearest neighbors , and set a threshold , if , then is considered a noise sample; if , then is considered a safe sample, otherwise is considered a dangerous sample; The set consisting of all safe samples is denoted by and the number of sample points is denoted by The set consisting of all dangerous samples is denoted by and the number of sample points is denoted by The set consisting of all noise samples is denoted by and the number of sample points is denoted by ​ Let be the minority class dataset after removing noisy samples, and , let be the number of samples in .

3. The software defect data generation method of claim 2, wherein: The majority class dataset is preprocessed to obtain a majority class dataset after removing noise , and based on a non-defect boundary data set is generated , the method comprises: In the software dataset finds each sample of the nearest neighbor samples, statistics the number of minority samples in the nearest neighbor samples ; For each sample , if , then is considered a noise sample; By about The set of all noise samples is denoted as The number of sample points of is denoted as Let be the majority class dataset after removing the noise samples, and the number of sample points of is ; In the data set find the nearest neighbor samples of each sample and put them all into the non-defect boundary data set .

4. The software defect data generation method of claim 3, wherein: The non-defect boundary data set is obtained by The security sample is calculated The selection probability of the security sample is calculated The method comprises the following steps: For each sample , the selection probability is calculated , as follows: ； wherein indicates the relative importance of a sample formula: ； representative sample non-defect boundary data set proximity to the non-defect boundary data set, given by: ； wherein, representing the sample and the proximity of the sample, the formula is: ； representing a sample between and euclidean distance; representative sample relative to a non-defect boundary dataset density, formula: ； representative sample with distance, formula ； wherein is a user-defined parameter.

5. The software defect data generation method of claim 4, wherein: said dangerous sample The cluster generates cluster sub-clusters and assigns weights to the cluster sub-clusters The method comprises: dangerous samples k-means clustering is performed to generate clusters of sub-clusters; Calculate the risk level of each cluster sub-cluster in the sample The relative weights in the formula are: ； wherein, represents the number of samples of the th cluster sub-cluster.

6. The software defect data generation method of claim 5, wherein: The number of the safety samples The number of the dangerous samples The number of the non-defect boundary data sets The selection probability and the weight The number of the new defect samples needed to be generated, the method comprises: According to the number of the safe samples and the dangerous samples , the number of the safe samples and the dangerous samples respectively needed to generate new samples and is calculated, according to the formula: ； ； Based on each cluster sub-cluster in relative weights Calculate the number of new samples needed to generate for each cluster sub-cluster. ,in, Indicates the first The number of new samples needed to generate each cluster sub-cluster is calculated using the following formula: ； For the safety sample New sample generation: first according to the selection probability Select a minority class safety sample in the set , set as ; Then randomly select a majority class boundary sample from the non-defect boundary data set , set as ; Perform feature interpolation on the two samples, the formula is: ； wherein, represents the first feature value of the newly generated sample, and the interpolation coefficient is randomly generated, and the step is repeated until the number of newly generated samples of the safe sample points reaches , and the safe sample is set as ; For dangerous samples New sample generation: for the defect dangerous data set The th cluster sub-cluster, the following operations are performed: randomly select two sample points , in the cluster sub-cluster, interpolate features according to the formula ; repeat this process until the number of newly generated samples in the cluster sub-cluster reaches , let the dangerous sample The newly generated sample set is ; Finally, a newly generated set of defect data samples is obtained .

7. The software defect data generation method of claim 5, wherein: If , then no matter what the value of is, only one cluster sub-cluster can be generated.

8. A software defect data generation system for implementing the software defect data generation method according to any one of claims 1 to 7, characterized by: The method for generating the required number of new defect samples by oversampling the defect data set and the non-defect data set in the step S2 comprises the following steps: The application relates to a method for generating new defect samples in a software defect data set, and a device thereof. The method comprises the following steps: An acquisition module is used for acquiring a software data set, wherein the software data set comprises a defect data set and a non-defect data set, the defect data set is taken as a minority class data set, and the non-defect data set is taken as a majority class data set; An oversampling module is used for oversampling the defect data set and the non-defect data set to generate the required new defect samples, and putting the new defect samples into the defect data set so that the sample points of the defect data set and the sample points of the non-defect data set reach balance after the new defect samples are put in.

Images