A software defect dataset classification method based on ensemble pruning
By integrating pruning technology, the problems of model overfitting and poor interpretability under complex data are solved, and the generalization ability and interpretability of the model are improved while maintaining the prediction performance, which is suitable for environments with limited resources.
Patent Information
- Application Number
- CN202310584439.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2023-05-23
- Publication Date
- 2025-10-03
- Estimated Expiration
- 2043-05-23
AI Technical Summary
Existing single learning algorithms or models have problems such as overfitting, underfitting, high bias or high variance when faced with complex data, resulting in unsatisfactory prediction performance. In addition, complex integrated models have poor model interpretability in fields such as finance, medicine, and law. It is necessary to improve the interpretability and explainability of the model while maintaining predictive performance.
A software defect dataset classification method based on ensemble pruning is adopted. The training set is generated by the bootstrap method, the validation set is obtained using the KNN algorithm, the dataset is constructed using the evolutionary clustering algorithm, and the base classifier is selected according to the principle of maximum accuracy. The redundant parts are pruned to reduce the model complexity and improve the generalization performance and robustness of the model.
It significantly reduces the number of base classifiers in the ensemble learning model, lowers computational overhead and storage requirements, improves the model's generalization performance and adaptability to unknown software defect data, saves energy, and is suitable for resource-limited environments.
Smart Images
Figure CN116578928B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to integrated learning, pruning, genetic algorithm and software defect prediction, and in particular to an integrated pruning method applying evolutionary clustering. Background Art
[0002] The rapid development of science and technology and the widespread use of the internet have led to the continuous generation, accumulation, and dissemination of massive, high-speed, and diverse data. The rapid accumulation and processing of big data has made data a vital asset for businesses and organizations, providing them with greater commercial value. By analyzing and mining big data, we can uncover patterns and trends hidden within the data, enabling more informed decisions.
[0003] A single learning algorithm or model may have limitations when used with complex data, such as overfitting, underfitting, high bias, or high variance. This can lead to suboptimal prediction performance or fail to meet actual needs. Ensemble learning combines multiple learning algorithms or models to achieve more accurate and robust predictions through joint decision-making, voting, and weighting. Ensemble learning leverages the complementarity between different models, improving model generalization and reducing the risk of overfitting. It also offers improved performance when handling large-scale data and complex tasks.
[0004] Ensemble pruning is a method that combines ensemble learning and model pruning techniques. It aims to improve the predictive performance and interpretability of the model by removing redundant and unnecessary parts in the ensemble model.
[0005] In practical applications, as data size and model complexity continue to increase, model interpretability and explainability are becoming increasingly important requirements. Complex ensemble models may offer excellent predictive performance, but their ability to explain the model's internal decision-making processes and feature importance is poor. This can pose limitations for scenarios requiring model interpretability, such as finance, healthcare, and law. Therefore, a method is needed to improve model interpretability and explainability while maintaining predictive performance and reducing the excessive energy consumption of big data processing. Summary of the Invention
[0006] In practical applications, model generalization and robustness are crucial. Complex ensemble models can suffer from overfitting, decreased generalization, and high sensitivity to input data. Pruning ensemble models can remove redundant and unnecessary model components, reducing model complexity and improving generalization and robustness. This can save computer power and protect computer storage.
[0007] In order to achieve the above object, the technical solution of the present invention is:
[0008] A software defect dataset classification method based on integrated pruning, characterized by comprising the following steps:
[0009] (1) Load the software defect dataset information. (2) Use the bootstrap method to obtain different training sets, and use all the datasets to train different base classifiers. (3) For the generation of the validation set, use the KNN algorithm to obtain a dataset that is similar to the test set in the training set as the validation set, and obtain the performance of each classifier. (4) The dataset of the evolutionary clustering algorithm is constructed by using the confidence of each base classifier on the validation set to construct the dataset. (5) Use the evolutionary clustering method to obtain the global optimal clustering result, and group all the base classifiers. (6) According to the principle of maximum accuracy, select the base classifier from each group. Reduce the running time and memory requirements of ensemble learning on the software defect dataset, and improve the adaptability to unknown software defect data.
[0010] Furthermore, the step (1) loads the input software defect dataset, first determines whether the dataset is a classification problem dataset, then collects statistics on the instance information, feature number information, and category information of the dataset, and finally converts the character type category into a numeric type according to one-hot encoding.
[0011] In step (2), bootstrap sampling is used to obtain different sub-training sets for training different base classifiers according to the number of base classifiers, and diversity among the base classifiers is ensured.
[0012] Furthermore, the steps of generating the validation set and obtaining the base classifier performance in step (3) are as follows:
[0013] Step 3.1: Set the hyperparameters of the KNN algorithm to obtain an appropriate number of validation set instances.
[0014] Step 3.2: Use the test set as the training set for the KNN algorithm, and use the trained KNN algorithm to test the original training set. If there are repeated instances in the training set, remove the duplicates and the data set becomes the validation set for the base classifier.
[0015] Step 3.3: After obtaining the validation sets of all base classifiers, each base classifier is used to predict the validation set, and its accuracy is calculated. The accuracy is sorted in descending order, and the top 50% base classifiers are selected for grouping.
[0016] Furthermore, the steps for constructing the evolutionary clustering data set in step (4) are as follows:
[0017] Step 4.1: Get the confidence scores of all base classifiers on the validation set. On the validation set, assuming there are N samples and the number of labels is M, for a base learner w on the validation set, the score structure is in The base learner w represents the probability that the i-th sample belongs to the j-th label. At this time, each score is formed into a two-dimensional array, which is then transformed into a one-dimensional array of the same size.
[0018] Step 4.2: After conversion into one-dimensional data, concatenate the confidence scores of all base classifiers into a two-dimensional array. The two-dimensional data at this time is the data set of the evolutionary clustering algorithm.
[0019] Furthermore, the steps of obtaining the global optimal clustering result using the evolutionary clustering method in step (5) are as follows:
[0020] Step 5.1: Set the hyperparameters of the evolutionary clustering algorithm, such as population size of 200, number of iterations of 200, and mutation probability of 0.5%.
[0021] Step 5.2: Initialize the population. Use the K-means method to train and predict the clustering dataset. The K-means clustering result is used as the chromosome of an individual. Based on this, a random function is then used to randomly select locations and randomly change the clustering result. The resulting population has significant differences in individual chromosomes, ensuring the diversity of the initial population.
[0022] Step 5.3: For the fitness calculation of the individuals in the population, a supervised classifier is used to calculate the fitness of each individual. A certain number of training sets are selected to train each supervised classifier. The F1 score of each supervised classifier on the test set is calculated and averaged. This average is used as the fitness of the individual.
[0023] Step 5.4: After calculating the fitness of the individuals in the initial population, use the bidding operator to select the parent generation.
[0024] Step 5.5: When using the single-point crossover operator for the parent individuals, the generated new individuals have the characteristics of the parents and are likely to produce better offspring under the condition of better fitness.
[0025] Step 5.6: After completing the single-point crossover, use the exchange mutation operator to improve the search ability and effect of the evolutionary clustering algorithm.
[0026] Step 5.7: After completing the three operators, eliminate the selected individuals according to the set number of iterations and select the individual with the largest fitness.
[0027] Step 5.8: Obtain the individual with the largest global fitness. According to this clustering result, group the base classifiers used in the evolutionary clustering algorithm data set.
[0028] Furthermore, the step (6) of selecting a base classifier from each group according to the principle of maximum accuracy is as follows:
[0029] Step 6.1: First, select the base classifier with the highest accuracy on the validation set from the first group and add it to the ensemble pruning model.
[0030] Step 6.2: Use sequential forward selection to select the base classifiers for the second group. Fuse each base classifier from the second group with the base classifiers from the ensemble pruned model and select the base classifier with the highest ensemble accuracy. Repeat this process for all groups.
[0031] Step 6.3: If the number of base classifiers in the ensemble pruned model still does not meet the ensemble pruned quantity after traversing all groups, start traversing again from the first group until the pruned quantity requirement is met.
[0032] The technical effects of the present invention are:
[0033] 1. By using ensemble pruning techniques, the number of base classifiers in an ensemble learning model applied to a software defect dataset can be significantly reduced. This is particularly useful for deploying models on mobile devices or in resource-constrained environments, as it reduces the model's storage requirements and computational overhead during inference, thereby reducing the power consumption of computer data processing and preserving computer storage devices.
[0034] 2. Improve the generalization performance of the ensemble learning model. Pruning can eliminate overfitting problems in the model, thereby improving the model's generalization ability and adaptability to unknown software defect data.
[0035] 3. Reduce the computational overhead of the ensemble learning model: Through pruning, the number of parameters and computational complexity of the ensemble learning model can be reduced, thereby reducing the computational overhead of the model, saving energy, and making the model more suitable for resource-limited environments.
[0036] 4. The present invention can reduce the running time and memory requirements of ensemble learning on software defect datasets, and can achieve performance similar to or even better than the original ensemble method under pruning conditions. BRIEF DESCRIPTION OF THE DRAWINGS
[0037] Figure 1 It is the overall system function diagram of the present invention;
[0038] Figure 2 Data preprocessing for the present invention;
[0039] Figure 3A flow chart for forming a genetic algorithm data set of the present invention;
[0040] Figure 4 This is a flow chart of the evolutionary clustering method of the present invention;
[0041] Figure 5 This is a flow chart of the population fitness calculation of the present invention;
[0042] Figure 6 This is a flow chart of the greedy algorithm of the present invention; DETAILED DESCRIPTION
[0043] The present invention will be described in detail below with reference to the various embodiments shown in the accompanying drawings. However, these embodiments do not limit the present invention, and any structural, methodological, or functional modifications made by those skilled in the art based on these embodiments are all within the scope of protection of the present invention.
[0044] like Figure 1 As shown, a software defect dataset classification method based on ensemble pruning includes: entering a software defect dataset, dividing the dataset into a training set and a test set, using KNN to obtain a validation set, selecting the top 50% base classifiers in the accuracy ranking, forming a dataset for an evolutionary clustering algorithm, training the evolutionary clustering method to obtain a clustering result with the maximum global fitness, grouping all base classifiers, and selecting the most representative base classifier from each group.
[0045] The method of the present invention mainly comprises the following parts:
[0046] 1. Software Defect Dataset Preprocessing
[0047] For the input software defect dataset, we first use the one-hot method to convert the character categories in the dataset into digital categories. After the conversion is completed, the software defect dataset is divided into a training set and a test set. For the training set, we use the bootstrap sampling method to obtain multiple different sub-training sets. For the test set, we use the test set as the training set of the KNN algorithm to obtain the neighbor instances of the test set on the training set, and then combine all the neighbor instances into a validation set. The process is as follows Figure 2 The specific steps are as follows:
[0048] Step 1: After parsing the software defect dataset, handle the missing values in the dataset and convert character categories into numeric categories;
[0049] Step 2: Use the replacement sampling method to generate a training set for each base classifier;
[0050] Step 3: Use the KNN algorithm to form the validation set. The test set is used as the training set for the KNN algorithm. The trained KNN algorithm is used to predict the training set, obtain the neighbors of the test set instances on the training set, and merge all the neighbors into the validation set.
[0051] 2. Dataset composition of evolutionary clustering method
[0052] For the trained base classifiers, the accuracy of each base classifier on the validation set is obtained, and the base classifier with the largest accuracy is selected. For the selected base classifiers, the confidence of each base classifier on the validation set is obtained and they are combined into a data set. The process is as follows Figure 3 The specific steps are as follows:
[0053] Step 1: Calculate the accuracy of each base classifier on the validation set, sort them in descending order of accuracy, and select the top 50% of the base classifiers to add to the set Q;
[0054] Step 2: Obtain the confidence of all base classifiers in set Q on the validation set. At this time, the confidence of each base classifier constitutes a two-dimensional array, which is converted into a one-dimensional array.
[0055] Step 3: After converting all base classifiers into one-dimensional arrays, they are concatenated into a two-dimensional array, which is the data set of the evolutionary clustering algorithm;
[0056] 3. Evolutionary clustering obtains the global optimal clustering result
[0057] Using the evolutionary clustering method, through population initialization, population evaluation, parent selection, crossover and mutation of each individual, and finally running according to the set number of iterations to select the population chromosome with the maximum fitness. The process is as follows Figure 4 The specific steps are as follows:
[0058] Step 1: Set the hyperparameters of the evolutionary clustering algorithm, such as population size, number of iterations, mutation rate, etc.
[0059] Step 2: Population initialization, using the K-means algorithm plus a random function to obtain multiple different population chromosomes and ensure diversity among the population;
[0060] Step 3: Population fitness evaluation, using multiple supervised classifiers to jointly calculate fitness. The original evolutionary clustering method dataset and the population chromosomes are spliced together to form a classification dataset. For this dataset, 50% is selected as the training set and 50% as the test set. After training multiple supervised classifiers, the F1 score of each supervised classifier on the test set is calculated, and the F1 scores of all supervised classifiers are added together to obtain the average. The average is the fitness score of the population. The method is as follows Figure 5 As shown;
[0061] Step 4: Calculate the fitness scores of all populations, use the competition operator to select the parent generation, and use crossover and mutation operations on the population of the parent generation to generate a new generation of population;
[0062] Step 5: Iterate and eliminate the population according to the set number of times, and finally obtain a population with the largest global fitness score. The chromosome of this population is the final clustering result;
[0063] 4. Base classifier selection
[0064] After getting the clustering results, all base classifiers are grouped and base classifiers are selected from each group. Using the exhaustive method takes too much time, so here we choose to use the greedy method to select base classifiers from each group. The process is as follows Figure 6 The specific steps are as follows:
[0065] Step 1: Based on the optimal clustering results, obtain each cluster and group the corresponding base classifiers according to the subscripts in each cluster;
[0066] Step 2: Select the base classifier with the highest accuracy from the first cluster and add it to the set E;
[0067] Step 3: Select the base classifier with the highest ensemble accuracy with all base classifiers in E from each remaining group and add it to the set E;
[0068] Step 4: Use the majority voting method to combine the base classifiers in the set E that meet the quantity requirement to predict software defect instances;
[0069] The present invention can reduce the running time and memory requirements of ensemble learning on software defect data sets, and can achieve performance similar to or even better than that of the original ensemble method under pruning conditions.
[0070] Throughout this specification, reference to terms such as "one embodiment," "some embodiments," "illustrative embodiments," "examples," "specific examples," or "some examples" means that a specific feature, structure, material, or characteristic described in conjunction with that embodiment or example is included in at least one embodiment or example of the present invention. In this specification, illustrative uses of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples.
[0071] While embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions, and variations may be made to the embodiments without departing from the principles and spirit of the invention, and that the scope of the invention is defined by the claims and their equivalents.
Claims
1. A software defect dataset classification method based on ensemble pruning, characterized by: The following steps are involved: (1) Loading software defect dataset information; (2) Use the bootstrap method to obtain different training sets, and use all the data sets to train different base classifiers; (3) For the generation of the validation set, the KNN algorithm is used to obtain a dataset that is similar to the test set in the training set as the validation set, and the performance of each classifier is obtained; (4) The dataset of the evolutionary clustering algorithm is constructed by using the confidence of each base classifier on the validation set to construct the dataset; (5) The evolutionary clustering method is used to obtain the global optimal clustering result and group all base classifiers; (6) According to the principle of maximum accuracy, the base classifier is selected from each group to reduce the running time and memory requirements of the ensemble learning on the software defect dataset, and improve the adaptability to unknown software defect data; The steps of selecting a base classifier from each group according to the maximum accuracy principle in step (6) are as follows: Step 6.1: First, select the base classifier with the highest accuracy on the validation set from the first group and add it to the ensemble pruning model; Step 6.2: For the base classifiers of the second group, use the sequential forward selection method to fuse each base classifier of the second group with the base classifiers in the ensemble pruning model, select the base classifier with the maximum ensemble accuracy, and repeat the same operation for all groups; Step 6.3: If the number of base classifiers in the ensemble pruned model still does not meet the ensemble pruned quantity after traversing all groups, start traversing again from the first group until the pruned quantity requirement is met.
2. The software defect dataset classification method based on integrated pruning according to claim 1, characterized in that: The step (1) loads the input software defect dataset, first determines whether the dataset is a classification problem dataset, then performs statistics on the instance information, feature number information, and category information of the dataset, and finally converts the character type category into a digital type according to one-hot encoding.
3. The software defect dataset classification method based on integrated pruning according to claim 1, characterized in that: In step (2), bootstrap sampling is used to obtain different sub-training sets for training different base classifiers according to the number of base classifiers, and diversity among the base classifiers is ensured.
4. The software defect dataset classification method based on integrated pruning according to claim 1, characterized in that: The steps of generating the validation set and obtaining the base classifier performance in step (3) are as follows: Step 3.1: Set the hyperparameters of the KNN algorithm to obtain an appropriate number of validation set instances. Step 3.2: Use the test set as the training set for the KNN algorithm, and use the trained KNN algorithm to test the original training set. If there are duplicate instances in the training set, remove the duplicates and the data set becomes the validation set for the base classifier. Step 3.3: After obtaining the validation sets of all base classifiers, each base classifier is used to predict the validation set, and its accuracy is calculated. The accuracy is sorted in descending order, and the top 50% base classifiers are selected for grouping.
5. The software defect dataset classification method based on integrated pruning according to claim 1, characterized in that: The steps of constructing the evolutionary clustering data set in step (4) are as follows: Step 4.1: Get the confidence scores of all base classifiers on the validation set. On the validation set, assuming there are N samples and the number of labels is M, for a base learner w on the validation set, the score structure is in Represents the probability of the base learner w being the i-th sample belonging to the j-th label; each score is formed into a two-dimensional array, which is then transformed into a one-dimensional array of the same size; Step 4.2: After conversion into one-dimensional data, concatenate the confidence scores of all base classifiers into a two-dimensional array. The two-dimensional data at this time is the data set of the evolutionary clustering algorithm.
6. The software defect dataset classification method based on integrated pruning according to claim 1, characterized in that: The steps of obtaining the global optimal clustering result using the evolutionary clustering method in step (5) are as follows: Step 5.1: Set the hyperparameters of the evolutionary clustering algorithm, such as population size of 200, number of iterations of 200, and mutation probability of 0.5%; Step 5.2: Initialize the population. Use the K-means method to train and predict the clustering data set. The clustering result obtained by K-means is used as the chromosome of an individual. Then, based on this, use a random function to randomly select positions and randomly change the cluster results. In the population obtained at this time, there are large differences in individual chromosomes, ensuring the diversity of the initial population. Step 5.3: For the fitness calculation of the individual in the population, a supervised classifier is used to calculate the fitness of each individual. A certain number of training sets are selected to train each supervised classifier. The F1 score of each supervised classifier on the test set is calculated and averaged. The average value will be used as the fitness of the individual. Step 5.4: After calculating the fitness of the individuals in the initial population, use the bidding operator to select the parent generation; Step 5.5: Use the single-point crossover operator on the parent individuals to generate new individuals with the characteristics of the parents. At the same time, under the condition of better fitness, it is possible to produce better offspring. Step 5.6: After completing the single-point crossover, use the exchange mutation operator to improve the search ability and effect of the evolutionary clustering algorithm; Step 5.7: After completing the three operators, eliminate the selected individuals according to the set number of iterations and select the individual with the largest fitness; Step 5.8: Obtain the individual with the largest global fitness. According to this clustering result, group the base classifiers used in the evolutionary clustering algorithm data set.