Software defect prediction method and system based on deep ensemble learning

Abstract

The invention discloses a software defect prediction method and system based on deep integrated learning, and belongs to the field of machine learning and software engineering. The method comprises the following steps: normalizing and dividing a software defect data set; expanding feature dimensions by using principal component analysis, and constructing graph shape data; balancing the training set by adopting an SMOTE technology; constructing a three-layer convolutional neural network comprising a residual block and an attention mechanism to extract deep features; after feature extraction of each layer, introducing a random forest as an auxiliary predictor for prediction, and dynamically calculating a weight according to the accuracy of the random forest for weighting; and finally, integrating weighted prediction results by taking XGBoost as a meta learning device, and outputting defect prediction. According to the method, the one-dimensional features are converted into the graph shape data, the potential relation between the features is mined through deep learning, and the software defect prediction accuracy and the model generalization ability are effectively improved in combination with the self-adaptive weighted integration strategy.

Description

Technical Field

[0001] This invention relates to the field of machine learning technology, and specifically to a software defect prediction method and system based on deep ensemble learning. Background Technology

[0002] As software systems grow in scale and complexity, hidden defects have become a key factor affecting their quality and reliability. Software defects can lead to serious security vulnerabilities, economic losses, and even catastrophic consequences. Therefore, proactively and accurately identifying potential defective modules during the software testing phase or before release is of great significance for the rational allocation of limited testing resources, control of development costs, and improvement of software quality.

[0003] Software defect prediction technology aims to build predictive models using historical metadata (such as code complexity and number of code changes) of software modules to determine whether new modules have a defect tendency. Traditional methods mainly rely on shallow machine learning models such as logistic regression, support vector machines, and decision trees. However, these methods have significant limitations: First, software metrics often have complex nonlinear relationships and potential correlations, making it difficult for shallow models to effectively mine and utilize this deep information; second, software defect data usually suffers from severe class imbalance (i.e., the number of non-defective samples far exceeds the number of defective samples), leading to skewed model prediction performance; finally, the generalization ability of a single model is limited, and its performance is unstable across different projects or datasets.

[0004] In recent years, deep learning methods, especially convolutional neural networks (CNNs), have achieved great success in fields such as image and speech recognition due to their powerful ability to automatically extract features and learn representations. Some studies have begun to explore their application in software defect prediction by reshaping one-dimensional feature vectors into two-dimensional matrices (such as in image form) and using CNNs to learn higher-order abstract patterns between features. However, directly applying CNNs to such structured data still faces challenges such as vanishing gradients, overfitting, and how to effectively fuse feature information from different levels. Meanwhile, ensemble learning methods (such as stacking) have been shown to effectively improve the generalization and robustness of models by combining the prediction results of multiple base learners. However, traditional ensemble strategies typically use simple averaging or voting on the outputs of base learners, failing to fully consider the contribution of the performance differences of different base learners to the final decision.

[0005] Therefore, existing technologies still suffer from problems such as insufficient mining of deep feature relationships, sensitivity to class imbalance, and room for improvement in model generalization ability. There is an urgent need for a novel software defect prediction scheme that can deeply integrate the advantages of deep feature extraction and adaptive ensemble learning. Summary of the Invention

[0006] The purpose of this invention is to overcome the shortcomings of existing technologies, such as insufficient feature mining ability, weak generalization ability, and sensitivity to imbalanced data in existing software defect prediction methods, and to provide a software defect prediction method and system based on deep ensemble learning.

[0007] In a first aspect, this invention proposes a software defect prediction method based on deep ensemble learning, comprising the following steps: S1. Normalize the original software defect dataset; S2. Divide the dataset processed by S1 into a training set for model training and a test set for performance evaluation. S3. Perform feature extraction and expansion on the training set and test set obtained by S2 to meet the feature dimensions required for constructing graph shape data; S4. Oversample the training set after processing in S3 to make the number of defective samples and non-defective samples in the training set the same. S5. Convert each sample data in the training set processed by S4 and the test set processed by S3 into a two-dimensional matrix of a preset size, as graph shape data. S6. Initialize the parameters of the convolutional neural network model, including the number of input channels and the kernel parameters; S7. Based on the parameters initialized in S6, construct a three-layer convolutional neural network; S8. Using the three-layer convolutional neural network constructed in S7, feature extraction is performed layer by layer on the graph shape data obtained in S5. After each layer of feature extraction, the extracted features are input into a corresponding random forest auxiliary predictor for prediction, resulting in multiple auxiliary prediction results and corresponding prediction accuracies. S9. Calculate the weights of each auxiliary predictor based on the prediction accuracy obtained in S8, and perform weighted fusion on the corresponding auxiliary prediction results. Input the weighted fusion result into the meta-learner, and the meta-learner outputs the final software defect prediction result.

[0008] Based on the above scheme, in S1, the original software defect dataset is processed using the max-min normalization method, and the specific formula is as follows: ; in, These are the original data values. It is the minimum value of this feature. It is the maximum value of this feature.

[0009] Based on the above scheme, in S2, a five-fold cross-validation method is used to divide the dataset. In each fold, the training set accounts for about 80% and the test set accounts for about 20%.

[0010] Based on the above scheme, in S3, the feature extraction and expansion specifically includes: using principal component analysis to extract the required number of principal component features, and concatenating the extracted principal component features to the original features, so that the total number of features for each sample reaches the number required to construct a two-dimensional matrix of a preset size.

[0011] Based on the above scheme, in S4, the oversampling process specifically involves: using synthetic minority class oversampling technology to generate new synthetic samples for the defective samples in the training set, until the number of defective samples in the training set is equal to the number of non-defective samples.

[0012] Based on the above scheme, in S5, the two-dimensional matrix of the preset size is an n×n square matrix, where n is an integer greater than 1.

[0013] Based on the above scheme, in S7, after the first and second convolutional layers of the three-layer convolutional neural network, two residual blocks are added respectively; each residual block integrates an attention mechanism module, which calculates attention weights through a 1×1 convolutional kernel and weights the output of the residual block.

[0014] Based on the above scheme, in S8, random forest is selected as an auxiliary predictor. After the convolutional neural network extracts features from different layers, random forest is used to predict the extracted features. Then, the accuracy of each auxiliary predictor is calculated and the weight is obtained. The calculation formula for the auxiliary predictor is: ; in, These represent the weights of the three auxiliary predictors. These represent the prediction accuracy of the three auxiliary predictors.

[0015] Based on the above scheme, in S9, the meta-learner is the XGBoost model.

[0016] Secondly, a software defect prediction system is provided, comprising: The data preprocessing module is used to perform steps S1 to S5, receive the original software defect dataset, and output the training set and test set in graph shape data format. The model building and training module is used in steps S6 to S8 to build and train a deep ensemble learning model containing a three-layer convolutional neural network and multiple random forest auxiliary predictors based on the graph shape training set data. The prediction execution module is used to execute the S9 step, which uses the trained deep ensemble learning model to process the graph shape test set data and output the software defect prediction result.

[0017] Thirdly, the present invention provides an electronic device including a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement the steps of the method described in the first aspect above.

[0018] Fourthly, the present invention provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps of the method described in the first aspect above.

[0019] The beneficial effects of this invention are: 1. By converting one-dimensional software metric features into two-dimensional graph shape data and using a three-layer convolutional neural network (CNN) combined with attention mechanism residual blocks for feature extraction, it can automatically and deeply mine the complex nonlinear relationships and potential patterns between features, overcoming the shortcomings of insufficient feature representation capabilities of traditional shallow models.

[0020] 2. Innovatively, a random forest is introduced as an auxiliary predictor after each layer of the CNN, and its weight is dynamically calculated based on the real-time performance (accuracy) of each auxiliary predictor, and the prediction results are weighted and fused. This mechanism gives the better-performing base learner more say in the final decision, improving the decision quality of the ensemble model.

[0021] 3. The Stacking ensemble learning framework is adopted, with the weighted output of the auxiliary predictor as the meta-feature, and the XGBoost meta-learner performs the final prediction. This effectively integrates the advantages of different levels and types of models, and significantly enhances the model's generalization ability and robustness when facing different data distributions.

[0022] 4. Before training, SMOTE technology is used to oversample defect samples in the training set, which balances the class distribution and enables the model to better learn the features of defect samples, thereby improving the recall rate of defect modules and the overall prediction performance.

[0023] 5. Experimental results on multiple publicly available software defect datasets show that the method of this invention outperforms the traditional Stacking ensemble learning method in multiple evaluation metrics such as F-measure, G-mean, and AUC, verifying its effectiveness and advancement. Attached Figure Description

[0024] Figure 1 This is a schematic diagram of a software defect prediction method based on deep ensemble learning according to the present invention; Figure 2 This is an overall flowchart of a software defect prediction method based on deep ensemble learning according to the present invention. Figure 3This is a parameter setting diagram of a software defect prediction method based on deep ensemble learning according to the present invention; Figure 4 These are the experimental results of the proposed method and the Stacking ensemble learning method on the derby-10.5.1.1 dataset; Figure 5 These are the experimental results of the proposed method and the Stacking ensemble learning method on the ActiveMQ-5.0.0 dataset. Detailed Implementation

[0025] To make the objectives, advantages and features of the present invention more apparent, the present invention will be further described in detail below with reference to specific embodiments.

[0026] Software defect prediction involves modeling historical data to predict whether a software module is prone to defects. A three-layer convolutional neural network (CNN) is a deep learning model consisting of an input layer, an output layer, a fully connected layer, and three convolutional layers. Auxiliary predictors, which extract features from each CNN layer, provide independent predictions using the prediction model, mitigating the gradient vanishing problem. This invention uses the prediction results of the three auxiliary predictors as input to a Stacking ensemble learning base predictor. Stacking ensemble learning is an ensemble learning method that includes a base learner and a meta-learner. This method first trains the base learner, then uses its output as meta-features as input to the meta-learner, and finally uses the meta-learner for prediction. The steps and conclusions of the method provided in this invention have been validated in PyCharm 2020.

[0027] like Figure 1 and Figure 2 As shown, to address the problems of difficulty in extracting effective features and weak generalization ability in traditional software defect prediction, this embodiment provides a software defect prediction method based on deep ensemble learning. In one specific embodiment, it may include the following steps: S1. Normalize the dataset.

[0028] A publicly available software defect dataset is obtained, containing metric features of multiple software modules (e.g., lines of code, cyclomatic complexity, code change frequency, etc.) and corresponding binary labels (1 indicating a defect, 0 indicating no defect). This invention uses min-max normalization to process the original dataset, achieving linear data scaling to eliminate dimensional differences. The specific formula is: ; in, These are the original data values. It is the minimum value of this feature. It is the maximum value of this feature.

[0029] S2. Use five-fold cross-validation to divide the dataset into training and test sets.

[0030] A five-fold cross-validation strategy was employed to evaluate the model's stability and generalization performance. The entire normalized dataset was randomly and uniformly divided into five folds. In each round of validation, one fold was selected alternately as the test set, and the remaining four folds were combined as the training set. This process was repeated five times to ensure that each sample was used as test data exactly once. In each fold, the training set comprised approximately 80% and the test set approximately 20%.

[0031] S3. Use principal component analysis (PCA) to extract principal components.

[0032] Publicly available software defect datasets consist of two parts: features and labels. Before performing convolution operations, the feature size should be [value missing]. The data contains graph-shaped data. When there are only a few features, it is impossible to construct. This invention uses PCA to extract a corresponding number of features and merges the extracted features with the original features to construct a matrix. The matrix we will construct Matrices are called graph-shaped data, specifically including: S3.1 Calculate the number of features required to expand the shape data of the composition diagram.

[0033] S3.2 Use PCA to extract the corresponding number of principal component features.

[0034] S3.3. Expand the features extracted by PCA to the right side of the training set and the test set respectively.

[0035] S4. Balance the training set using the Synthetic Minority Oversampling Technique (SMOTE).

[0036] During model training, the number of defect-free samples far exceeds the number of defective samples, causing the model to favor defect-free samples during prediction and reducing its performance on defective samples. Therefore, class imbalance is a key factor affecting the performance of software defect prediction models. To address this issue, this invention performs SMOTE oversampling on defective samples in the training set, ensuring that the number of defective and defect-free samples in the training set is equal, thereby improving the model's prediction performance on defective samples.

[0037] S5. Create a graph for each sample.

[0038] To utilize the powerful feature extraction capabilities of convolutional neural networks, one-dimensional samples need to be transformed into... The 2D shape. Therefore, this invention converts the training and test set samples into graph-shaped data, which are then used as input to a three-layer convolutional neural network. That is, the training set, after PCA augmentation and balancing, is converted into... The graph shape data (2D shape) is used as the input to the convolutional neural network.

[0039] S6. Initialize the model and related parameters.

[0040] Initialize the convolutional neural network's input channel count, output channel count, kernel size, stride, zero-padding count, and dropout rate. Specific parameter settings in this embodiment are as follows: Figure 3 As shown, conv represents the number of layers in the convolutional neural network, and 3x3 represents the kernel size.

[0041] S7. Construct a three-layer convolutional neural network.

[0042] This embodiment uses a convolutional neural network (CNN) to extract data features, employing three convolutional layers. The activation function used is ReLU. To prevent overfitting, a Dropout layer is added to the third CNN layer. Two residual blocks are added after the first and second CNN layers to learn deeper features and mitigate the vanishing gradient problem.

[0043] This invention uses an attention mechanism in each residual block, calculating attention weights using a 1x1 convolution kernel and weighting the residual block output. Specifically, it includes: S7.1 Construct a three-layer convolutional neural network, and add a Dropout layer after the third convolutional neural network layer.

[0044] S7.2 Add two residual blocks after the first and second convolutional neural networks, respectively.

[0045] S7.3. Use an attention mechanism in each residual block to compute attention weights through a 1x1 convolutional layer.

[0046] S8. After extracting features in each layer of the convolutional neural network, use random forest as an auxiliary predictor for prediction.

[0047] After extracting features from different layers of the convolutional neural network, this invention selects a random forest (RF) as an auxiliary predictor to independently predict the extracted features. Then, the prediction accuracies of the three auxiliary predictors are converted into weights, and the prediction results of the auxiliary predictors are used as input to the meta-learner. Specifically, this includes: S8.1. Use the features extracted by each layer of the convolutional neural network as inputs to the three auxiliary predictors.

[0048] S8.2, Use random forest for independent prediction.

[0049] S8.3. The results of the three auxiliary predictors are weighted according to their prediction accuracy, and the weighted prediction results are used as input to the meta-learner. The formula for calculating the weights of the three auxiliary predictors is as follows: ; in, These represent the weights of the three auxiliary predictors. These represent the prediction accuracy of the three auxiliary predictors.

[0050] S9. Select XGBoost as the meta-learner, construct a deep ensemble learning model, and output the prediction results. This invention selects XGBoost as the meta-learner, inputs the weighted prediction results from the auxiliary predictor, and then uses XGBoost to output the final software defect prediction results.

[0051] Based on the same inventive concept, this invention also provides an embodiment of a software defect prediction system. This system implements the steps in the aforementioned method embodiments, and its core lies in encapsulating the data preprocessing, model training, and prediction processes through modular design to provide efficient software defect prediction functionality. Figure 1 and Figure 2 As shown, the system may include the following modules: The data preprocessing module receives the raw software defect dataset and sequentially performs the following processing steps to output a training set and a test set in graph-shaped data format: Perform max-min normalization on the original dataset; Five-fold cross-validation was used to divide the normalized dataset into training and test sets. Principal component analysis (PCA) was performed on the split training and test sets to extract and augment features, ensuring that the feature dimensions of each sample met the requirements for constructing the test set. Requirements for graph shape data; SMOTE oversampling is performed on the training set to balance the number of defective and non-defective samples; Each sample data in the processed training and test sets is converted into a two-dimensional matrix, i.e., graph-shaped data.

[0052] The model building and training module, connected to the data preprocessing module, is used to receive graph-shaped training set data and perform the following operations to build and train a deep ensemble learning model: Initialize the parameters of the three-layer convolutional neural network, including the number of input channels, kernel size, stride, number of zero-padding, and dropout rate; Construct a three-layer convolutional neural network containing three convolutional layers and introducing residual blocks with attention mechanisms after the first and second layers; The convolutional neural network is used to extract features layer by layer from the graph shape training set, and the features extracted from each layer are input into a corresponding random forest auxiliary predictor for prediction. The weights of each auxiliary predictor are dynamically calculated based on their prediction accuracy, and the weighted prediction results are output as meta-features.

[0053] The prediction execution module, connected to the model building and training module, is used to load the trained deep ensemble learning model and predict defects in new software modules. Receive the raw measurement data of the software module to be predicted; The processing flow in the data preprocessing module is invoked to convert the data to be predicted into two-dimensional graph shape data; Input the graph shape data into the pre-trained convolutional neural network, extract three layers of features, and input them into the corresponding random forest auxiliary predictors respectively; The outputs of the auxiliary predictor are weighted and fused according to their weights, and the fused features are then input into the trained XGBoost meta-learner. The output meta-learner's prediction results determine whether the software module has defects.

[0054] The functions of each module in the above system embodiment correspond to the steps in the method embodiment. Through data transfer and invocation between modules, an automated process from raw data input to defect prediction result output is achieved. This system can be deployed on a server or local computing device to assist software testers in defect prediction and test resource allocation.

[0055] It should be understood that the modules in the above system embodiments can be implemented in software, hardware, or a combination of both. For example, each module can correspond to different functions or classes in a computer program, or it can be integrated into a dedicated image processing or machine learning acceleration chip. Those skilled in the art can adapt and optimize it according to the actual deployment environment.

[0056] To verify the performance of the method provided in this invention, we randomly selected two publicly available software defect datasets, derby-10.5.1.1 and activemq-5.0.0, for experiments. The comparison method uses the same preprocessed data as this invention, but the data used in the comparison method is not plotted. To ensure the fairness of the experiment, the comparison method uses three random forests as base predictors and XGBoost as the meta-learner. The experimental results are shown in Table 1: Table 1 ; Figure 4 and Figure 5These are the experimental results of the proposed method and the Stacking ensemble learning method on the publicly available software defect datasets derby-10.5.1.1 and activemq-5.0.0.

[0057] Analysis Table 1 Figure 4 and Figure 5 It can be concluded that the method proposed in this invention outperforms the Stacking ensemble learning method in terms of F-measure, G-mean, and AUC. Experimental results show that the deep ensemble method proposed in this invention can effectively extract the latent relationships between features and improve the generalization ability of the model.

[0058] It should be noted that the logic and / or steps in the embodiments, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a processor-included system, or other system that can fetch and execute instructions from, an instruction execution system, apparatus, or device). For the purposes of this specification, "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transmit programs for use by, or in conjunction with, an instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of computer-readable media include: an electrical connection having one or more wires (electronic device), a portable computer disk drive (magnetic device), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic devices, and portable optical disc read-only memory (CDROM). Alternatively, the computer-readable medium may be paper or other suitable media on which the program can be printed, since the program can be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, interpreting, or otherwise processing as necessary, and then stored in a computer memory.

[0059] It should be understood that various parts of the present invention can be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented in software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0060] Those skilled in the art will understand that all or part of the steps of the methods described in the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, the program includes one or a combination of the steps of the method embodiments.

[0061] Furthermore, in the embodiments of the present invention, the functional modules can be integrated into one processing module, or each module can exist physically separately, or two or more modules can be integrated into one module. The integrated module can be implemented in hardware or as a software functional module. If the integrated module is implemented as a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. The aforementioned storage medium can be a read-only memory, a hard disk, or an optical disk, etc.

[0062] The above embodiments have provided a detailed description of the technical solutions of the present invention. Obviously, the present invention is not limited to the described embodiments. Based on the embodiments of the present invention, those skilled in the art can make various changes, but any changes that are equivalent or similar to the present invention fall within the scope of protection of the present invention. Contents not described in detail in this specification are prior art known to those skilled in the art.

Claims

1. A software defect prediction method based on deep ensemble learning, characterized in that, Includes the following steps: S1. Normalize the original software defect dataset; S2. Divide the dataset processed by S1 into a training set for model training and a test set for performance evaluation. S3. Perform feature extraction and expansion on the training set and test set obtained by S2 to meet the feature dimensions required for constructing graph shape data; S4. Oversample the training set after processing in S3 to make the number of defective samples and non-defective samples in the training set the same. S5. Convert each sample data in the training set processed by S4 and the test set processed by S3 into a two-dimensional matrix of a preset size, as graph shape data. S6. Initialize the parameters of the convolutional neural network model, including the number of input channels and the kernel parameters; S7. Based on the parameters initialized in S6, construct a three-layer convolutional neural network; S8. Using the three-layer convolutional neural network constructed in S7, feature extraction is performed layer by layer on the graph shape data obtained in S5. After each layer of feature extraction, the extracted features are input into a corresponding random forest auxiliary predictor for prediction, resulting in multiple auxiliary prediction results and corresponding prediction accuracies. S9. Calculate the weights of each auxiliary predictor based on the prediction accuracy obtained in S8, and perform weighted fusion on the corresponding auxiliary prediction results. Input the weighted fusion result into the meta-learner, and the meta-learner outputs the final software defect prediction result.

2. The method according to claim 1, characterized in that, In S1, the original software defect dataset is processed using the max-min normalization method, with the specific formula as follows: ； in, These are the original data values. It is the minimum value of this feature. It is the maximum value of this feature.

3. The method according to claim 1, characterized in that, In S2, a five-fold cross-validation method is used to split the dataset. In each fold, the training set accounts for 80% and the test set accounts for 20%.

4. The method according to claim 1, characterized in that, In S3, the feature extraction and expansion specifically includes: using principal component analysis to extract the required number of principal component features, and then concatenating the extracted principal component features to the original features, so that the total number of features for each sample reaches the number required to construct a two-dimensional matrix of a preset size.

5. The method according to claim 1, characterized in that, In S4, the oversampling process specifically involves: using synthetic minority class oversampling technology to generate new synthetic samples from the defective samples in the training set until the number of defective samples in the training set is equal to the number of non-defective samples.

6. The method according to claim 1, characterized in that, In S5, the two-dimensional matrix of the preset size is an n×n square matrix, where n is an integer greater than 1.

7. The method according to claim 1, characterized in that, In S7, after the first and second convolutional layers of the three-layer convolutional neural network, two residual blocks are added respectively; each residual block integrates an attention mechanism module, which calculates attention weights through a 1×1 convolutional kernel and weights the output of the residual block.

8. The method according to claim 1, characterized in that, In S8, random forest is selected as the auxiliary predictor. After the convolutional neural network extracts features from different layers, random forest is used to predict the extracted features. Then, the accuracy of each auxiliary predictor is calculated and the weight is obtained. The calculation formula for the auxiliary predictor is: ； in, These represent the weights of the three auxiliary predictors. These represent the prediction accuracy of the three auxiliary predictors.

9. The method according to claim 1, characterized in that, In S9, the meta-learner is the XGBoost model.

10. A software defect prediction system, characterized in that, include: A data preprocessing module is used to perform steps S1 to S5 as described in any one of claims 1 to 9, receive the original software defect dataset, and output a training set and a test set in graph-shaped data format; The model building and training module is used to perform steps S6 to S8 of any one of claims 1 to 9, and to build and train a deep ensemble learning model containing a three-layer convolutional neural network and multiple random forest auxiliary predictors based on the graph shape training set data. The prediction execution module is used to execute step S9 of any one of claims 1 to 9, process the graph shape test set data using the trained deep ensemble learning model, and output the software defect prediction result.

Images