Integrated genetic feature selection method based on attention mechanism optimization

Abstract

The invention discloses an attention mechanism optimization-based integrated genetic feature selection method, which is used for carrying out feature selection on a training data set for software defect prediction through a genetic algorithm. Specifically, a binary system is used for coding genes of individuals, then the individuals are used as masks, features are selected from a training data set to construct a feature subset, 1 represents feature selection, and 0 represents feature non-selection. The mass fraction of each gene in the parent is calculated through a multi-head attention mechanism, the crossed positions are selected according to the mass fractions of the genes to generate offspring individuals, and the higher the mass fractions of the genes are, the higher the probability of being selected is. Training the base model by using the feature subset, calculating a mean square error of the feature subset on the base model, taking a negative value as a fitness value, taking an individual with the highest fitness value as an optimal individual, selecting features from the training data set, and constructing the feature subset to train the software defect prediction model. And the generalization ability and the training efficiency of the model are improved.

Description

Technical Field

[0001] This invention belongs to the field of electronic digital data processing technology, and relates to software defect prediction, specifically to an integrated genetic feature selection method based on attention mechanism optimization. Background Technology

[0002] As software systems continue to grow in size and complexity, software defect prediction has become a critical step in ensuring software quality. Software defects not only lead to a decline in user experience but can also cause serious problems such as system crashes, data loss, and security vulnerabilities. Therefore, accurately predicting potential defective modules in the early stages of software development and rationally allocating limited testing and remediation resources is of paramount importance.

[0003] Traditional feature selection methods, such as ReliefF, Information Gain, and Chi-square test, select the most valuable features for software defect prediction by evaluating the correlation between features and the target variable. These are all filtering feature selection methods, simple and efficient, but may ignore the interactions between features. In addition, there are feature selection methods specifically for predicting the number of software defects, such as HFSNFP, FSCR, and FSDNP, which select features through multiple dimensions. However, existing feature selection algorithms are often limited to linear relationship analysis between features, ignoring nonlinear relationships. This may lead to inaccurate feature selection, thus affecting the performance of the prediction model. Secondly, traditional algorithms are often computationally inefficient in high-dimensional feature spaces and are prone to getting trapped in local optima, making it difficult to find the optimal feature subset. Finally, some feature selection methods are slow when processing large-scale datasets, failing to meet the efficiency requirements of practical applications.

[0004] Genetic algorithms are global optimization algorithms that simulate the biological evolutionary process. They continuously evolve a population through operations such as selection, crossover, and mutation to find the optimal solution to a problem. In feature selection problems, genetic algorithms can efficiently explore the feature space and find the optimal subset of features. However, traditional genetic algorithms may face problems of low computational efficiency and susceptibility to getting trapped in local optima when dealing with high-dimensional feature spaces. Summary of the Invention

[0005] To address the shortcomings of existing technologies, this invention proposes an integrated genetic feature selection method based on attention mechanism optimization. This method combines the global search capability of genetic algorithms with the local feature recognition capability of Transformer attention mechanism to evaluate the importance of features, construct a multi-objective dynamic screening mechanism for code semantic features, improve the efficiency of feature selection, and solve the problems of feature redundancy, insufficient semantic representation, and inefficient resource allocation in traditional software defect prediction.

[0006] An integrated genetic feature selection method based on attention mechanism optimization includes the following steps:

[0007] Step 1: Collect training data

[0008] The characteristics and corresponding defects of multiple software modules are obtained from the software development project and used as a training dataset.

[0009] Step 2: Initialize the genetic algorithm

[0010] Define the population size and the number of iterations to terminate the genetic algorithm, and set the iteration to stop when the number of iterations reaches the number of iterations to terminate the algorithm.

[0011] The number of genes in an individual is set to be the same as the number of features in the software module. Genes of individual m are encoded using binary, with 1 indicating feature selection and 0 indicating feature deselection. An initial population is randomly generated, and the number of iterations is initialized to g=1.

[0012] As a preferred option, the number of genes encoding 1 in an individual is set to k.

[0013] Step 3: Dynamic Adaptive Assessment

[0014] Features are selected from the training dataset using individuals in the population to construct a feature subset. The base model is trained using the feature subset, and the mean squared error of the feature subset on the base model is calculated. The negative value is then used as the fitness value of the corresponding individual.

[0015] As a preferred method, feature subsets are used to train different base models, the mean square error of the feature subsets on each base model is calculated, the average value is calculated, and the negative value is taken as the fitness value of the corresponding individual.

[0016] Preferably, the base model includes Bayesian regression model, linear regression model, decision tree regression model, and support vector machine regression model.

[0017] Step 4: Crossover genetic iteration

[0018] Two parent individuals are randomly selected and concatenated to form a two-dimensional parent matrix. The multi-head attention mechanism of the Transformer encoder is used to calculate the quality score of each gene in the parent matrix. The crossover position is selected based on the quality score of the gene; the higher the quality score, the higher the probability of being selected. Offspring individuals are generated through crossover inheritance.

[0019] Repeat the crossover genetic operation P times to generate P offspring, then set the iteration count g+1 and perform the next crossover genetic iteration until the termination condition of the iteration is met.

[0020] As a preferred approach, positional encoding enhancement is performed on each row of the two-dimensional parent matrix before calculating the quality score of the genes.

[0021] As a preferred option, the top k genes with the highest quality scores from the parent matrix are selected as genes encoded as 1 in the offspring individuals, and the remaining genes are encoded as 0.

[0022] Step 5: Feature Selection

[0023] For the P individuals obtained through genetic iteration in step 4, select the individual with the highest fitness value as the optimal mask m*. Use the optimal mask m* to select features from the training dataset and construct a feature subset to train the software defect prediction model.

[0024] Compared with existing technologies, the present invention has the following beneficial effects.

[0025] In software defect prediction tasks, the features of software modules often have complex interrelationships. By introducing the Transformer's multi-head attention mechanism, the quality of parent genes is scored, and high-scoring gene loci are retained first. The rules of cross-genesis are upgraded to an attention-driven adaptive strategy, which dynamically adjusts the retention probability of parent features. This allows for the quantification of the marginal contribution of each feature to defect prediction, thereby extracting more discriminative feature representations, removing data noise caused by redundant features, avoiding overfitting, and improving the generalization ability and training efficiency of the software defect prediction model. Ultimately, this enhances the accuracy and interpretability of defect prediction. Attached Figure Description

[0026] Figure 1 This is a schematic diagram of an integrated genetic feature selection method based on attention mechanism optimization.

[0027] Figure 2 This is a flowchart of the crossover genetic iteration process;

[0028] Figure 3 This is a schematic diagram of dynamic crisscross inheritance. Detailed Implementation

[0029] The invention will be further explained below with reference to the accompanying drawings;

[0030] An attention-based optimization method for ensemble genetic feature selection, such as... Figure 1 As shown, the specific steps include:

[0031] Step 1: Data Collection and Preprocessing

[0032] We extracted 20-dimensional feature vectors from 15 historical versions of four open-source projects: Apache Ant, Camel, Ivy, and Xerces. These vectors included the number of lines of code, complexity, comment ratio, and number of functions. We also labeled the number of defects in each software module and represented them in matrix form. Each row of the matrix corresponds to the feature vector of a software module, and the last column represents the number of defects. This process was used to construct the original dataset.

[0033] Step 2: Initialize the genetic algorithm

[0034] like Figure 2 As shown, the population size is set to P=50, the number of iterations to terminate is G=10, and the number of iterations to initialize is g=1.

[0035] The genes of individual m are encoded using binary, where 1 indicates a selected feature and 0 indicates a deselected feature. The length d of individual m is 20, corresponding to the feature vector dimension of the software module. The number of 1s in the genes of individual m is k, and the value of k is determined based on the number of target features. In this embodiment, k=6.

[0036] Step 3: Dynamic Adaptive Assessment

[0037] Features are selected from the original dataset using individual m to construct a feature subset. Ten-fold cross-validation is then performed on multiple base models in the model set M using each feature subset. The mean squared error (MSE) between the software defect prediction result and the number of labeled defects is calculated for each base model. model Finally, the negative of the mean squared error of the feature subset on all base models is taken as the fitness (m) of individual m.

[0038] In this embodiment, the base models include Bayesian regression models, linear regression models, decision tree regression models, and support vector machine regression models.

[0039] Step 4: Crossover genetic iteration

[0040] s4.1, such as Figure 3 As shown, two parent individuals are randomly selected and spliced ​​together to form a parent matrix Xparents of size 2×20.

[0041] s4.2. Introduce positional encoding (PE) to enhance the positional encoding of each row of the parent matrix Xparents, resulting in the enhanced encoding matrix Xpc:

[0042] Xpc = Xparents + PE

[0043] The coding enhancement matrix Xpc is input into a lightweight Transformer encoder, and the quality score of each gene in the coding enhancement matrix Xpc is calculated using a multi-head attention mechanism.

[0044] Select the k genes with the highest scores from Xpc and assign them as 1 in the offspring individuals. The remaining genes are assigned as 0. This process generates offspring individuals and completes the crossover operation.

[0045] s4.3, repeat s4.1 and s4.2 until P offspring are generated. Let the iteration count g+1, and return to s4.1 until g=G.

[0046] Step 5: Feature Selection

[0047] For the P individuals obtained through genetic iteration in step 4, select the individual with the highest fitness value as the optimal mask m*. Use the optimal mask m* to select features from the original dataset and construct a feature subset for training the software defect prediction model.

[0048] To demonstrate the effectiveness of this method, feature selection was performed on different datasets using Relief, info, chi2, HFSNFP, FSCR, FSDNP, and MIC-TSFS. Ten-fold cross-validation was conducted on Bayesian ridge regression optimized models, linear regression models, and decision tree regression models using the full feature dataset and feature subsets obtained from different feature selection methods. AAE (Absolute Mean Error), ARE (Absolute Relative Error), and G-means (Geometric Mean) were used as evaluation metrics to assess the improvement of the software defect prediction model performance by the feature subsets selected by different methods. AAE is the average of all errors in a set of prediction data, indicating the gap between the predicted and actual results. ARE indicates the magnitude of the relative error when the overall target item is evaluated. G-means measures the accuracy of the prediction.

[0049] Tables 1-3 show the AAE, ARE, and G-means results of 10-fold cross-validation on the Bayesian Ridge Regression optimized model, using the full feature dataset and feature subsets obtained by different feature selection methods:

[0050] Table 1

[0051] full Relief info chi2 HFSNFP FSCR FSDNP MIC-TSFS This method ant-1.3 0.3013 0.2872 0.2949 0.3019 0.2788 0.2878 0.3026 0.2705 0.2391 ant-1.4 0.2814 0.2817 0.2647 0.2873 0.2761 0.1296 0.2647 0.2706 0.2689 ant-1.5 0.1157 0.1231 0.1195 0.1193 0.1129 0.1197 0.1336 0.1162 0.1229 ant-1.6 0.4817 0.4475 0.4617 0.4618 0.5332 0.4617 0.4642 0.4560 0.4521 ant-1.7 0.3705 0.5958 0.4041 0.3987 0.4631 0.4591 0.4228 0.3974 0.4311 camel-1.0 0.0442 0.0442 0.0442 0.0442 0.0442 0.0413 0.0442 0.0593 0.0413 camel-1.2 1.0389 1.0606 1.0326 1.0750 1.0914 1.0404 1.0488 1.0274 1.0444 camel-1.4 0.4771 0.5069 0.4633 0.4633 0.4828 0.4943 0.4783 0.4656 0.5057 camel-1.6 0.7044 0.7032 0.6888 0.7137 0.6959 0.7165 0.7003 0.6878 0.6835 xerces-1.2 0.2818 0.2614 0.2659 0.2818 0.2705 0.2682 0.275 0.2773 0.2773 xerces-1.3 0.5451 0.5445 0.5273 0.5668 0.5399 0.5358 0.5973 0.5361 0.5618 ivy-1.1 1.3121 1.3765 1.2576 1.3129 1.372 1.8402 1.3129 1.2394 1.2015 ivy-1.4 0.0827 0.0702 0.0952 0.0827 0.0785 0.0743 0.0827 0.0972 0.0788 ivy-2.0 0.1732 0.1562 0.1647 0.1705 0.1788 0.1647 0.1675 0.1618 0.1479 mean 0.4486 0.4613 0.4346 0.4436 0.4584 0.4738 0.4496 0.4330 0.4326

[0052] Table 2

[0053] full Relief info chi2 HFSNFP FSCR FSDNP MIC-TSFS This method ant-1.3 0.1728 0.1521 0.1556 0.1723 0.1495 0.1393 0.1633 0.1354 0.1194 ant-1.4 0.1351 0.1532 0.1334 0.1467 0.1447 0.0679 0.1185 0.1406 0.1413 ant-1.5 0.0725 0.0695 0.0677 0.0737 0.061 0.0678 0.0725 0.066 0.0637 ant-1.6 0.2829 0.2702 0.2775 0.272 0.3173 0.2697 0.2766 0.2733 0.2636 ant-1.7 0.2043 0.3834 0.2331 0.2259 0.2869 0.2844 0.2535 0.2255 0.2603 camel-1.0 0.0226 0.0226 0.0226 0.0226 0.0226 0.0197 0.0226 0.0407 0.0197 camel-1.2 0.6194 0.6503 0.6152 0.6554 0.6777 0.6352 0.6241 0.5957 0.6334 camel-1.4 0.2552 0.2876 0.2405 0.2402 0.2726 0.2872 0.2651 0.2421 0.2932 camel-1.6 0.3962 0.3756 0.3648 0.3884 0.3607 0.3876 0.3878 0.3674 0.3682 xerces-1.2 0.1225 0.0964 0.1092 0.1225 0.1055 0.1077 0.1127 0.118 0.1180 xerces-1.3 0.3133 0.2964 0.307 0.3171 0.2985 0.2697 0.3654 0.3127 0.3379 ivy-1.1 0.6571 0.6935 0.6401 0.6651 0.6106 0.8445 0.6651 0.6225 0.5437 ivy-1.4 0.0424 0.033 0.0549 0.0424 0.0382 0.0372 0.0424 0.0609 0.0415 ivy-2.0 0.0998 0.0891 0.0943 0.1043 0.1038 0.092 0.0959 0.0926 0.0841 mean 0.2463 0.2552 0.2369 0.2426 0.2464 0.2507 0.2475 0.2352 0.2349

[0054] Table 3

[0055] full Relief info chi2 HFSNFP FSCR FSDNP MIC-TSFS This method ant-1.3 0.4608 0.5099 0.451 0.5354 0.601 0.3223 0.4868 0.4606 0.6359 ant-1.4 0.0 0.2591 0.2372 0.1167 0.2131 0.5079 0.0 0.1892 0.2595 ant-1.5 0.4242 0.251 0.2679 0.4336 0.274 0.2679 0.0577 0.3127 0.1132 ant-1.6 0.7782 0.7815 0.7822 0.7809 0.7309 0.7716 0.7735 0.7874 0.7923 ant-1.7 0.7694 0.6668 0.7465 0.7427 0.7379 0.739 0.7509 0.7471 0.7475 camel-1.0 0 0.0 0.0 0.0 0.0 0.0 0.0 0.1054 0.0000 camel-1.2 0.5398 0.4266 0.5331 0.4961 0.2917 0.3907 0.5273 0.5363 0.5081 camel-1.4 0.5888 0.5675 0.5617 0.5768 0.6159 0.6275 0.5792 0.5693 0.6091 camel-1.6 0.5985 0.579 0.5872 0.5873 0.5641 0.4544 0.5968 0.5999 0.6171 xerces-1.2 0.1239 0.0 0.2089 0.1239 0.0 0.1163 0.0744 0.1245 0.1608 xerces-1.3 0.6364 0.7576 0.5902 0.5613 0.7591 0.6414 0.5643 0.6481 0.5816 ivy-1.1 0.3332 0.3053 0.5338 0.2923 0.6775 0.3236 0.2923 0.5558 0.5433 ivy-1.4 0 0.0707 0.0 0.0 0.0 0.0707 0.0 0.145 0.0691 ivy-2.0 0.3726 0.3641 0.3309 0.467 0.3113 0.3004 0.3858 0.3485 0.6114 mean 0.4081 0.3956 0.4165 0.4018 0.4126 0.3953 0.3635 0.4378 0.4464

[0056] As can be seen from Tables 1-3, the average AAE value of this method is 0.4326, the average ARE value is 0.2349, and the average G-Mean value is 0.2349. This method outperforms other methods on most datasets, especially on the ant-1.3 and ivy-2.0 datasets, indicating that the feature subset selected by this method has the highest prediction accuracy in the Bayesian Ridge Regression model.

[0057] Tables 4-6 show the AAE, ARE, and G-means results of 10-fold cross-validation on a linear regression model using the full feature dataset and feature subsets obtained by different feature selection methods:

[0058] Table 4

[0059] full Relief info chi2 HFSNFP FSCR FSDNP MIC-TSFS This method ant-1.3 0.3103 0.3045 0.3128 0.3115 0.2788 0.2885 0.2551 0.3038 0.2647 ant-1.4 0.3147 0.2869 0.3029 0.2703 0.2761 0.1667 0.2647 0.3029 0.2804 ant-1.5 0.1091 0.1367 0.1263 0.1193 0.1129 0.1264 0.1301 0.1194 0.1193 ant-1.6 0.4844 0.459 0.4759 0.476 0.5332 0.4644 0.4699 0.4616 0.4461 ant-1.7 0.3988 0.5957 0.4229 0.4148 0.4631 0.4483 0.4215 0.4189 0.4164 camel-1.0 0.0442 0.0442 0.0442 0.0442 0.0442 0.0413 0.0442 0.0702 0.0413 camel-1.2 1.034 1.0475 1.0275 1.0487 1.0914 1.0421 1.057 1.0701 1.0773 camel-1.4 0.4967 0.5069 0.4702 0.4817 0.4828 0.4886 0.4851 0.4944 0.4931 camel-1.6 0.7085 0.7053 0.7022 0.7086 0.6959 0.7144 0.7045 0.7002 0.6907 xerces-1.2 0.3091 0.2614 0.3045 0.2727 0.2705 0.275 0.2727 0.3023 0.2750 xerces-1.3 0.574 0.5244 0.5095 0.5779 0.5399 0.5556 0.6308 0.4984 0.5882 ivy-1.1 1.5485 1.3114 1.2856 1.3561 1.372 1.7477 1.3568 1.3212 1.2273 ivy-1.4 0.0785 0.0702 0.091 0.0827 0.0785 0.0743 0.0827 0.1048 0.0705 ivy-2.0 0.1703 0.1703 0.1617 0.1761 0.1788 0.1675 0.1646 0.1816 0.1565 mean 0.4700 0.4589 0.4455 0.4528 0.4581 0.4715 0.4528 0.4535 0.4390

[0060] Table 5

[0061] full Relief info chi2 HFSNFP FSCR FSDNP MIC-TSFS This method ant-1.3 0.1852 0.179 0.1832 0.1915 0.1495 0.1534 0.1348 0.1782 0.1457 ant-1.4 0.2018 0.1628 0.1874 0.1544 0.1447 0.1049 0.1185 0.1902 0.1566 ant-1.5 0.0692 0.0813 0.0761 0.0743 0.061 0.0729 0.0714 0.0727 0.0642 ant-1.6 0.2863 0.2734 0.2855 0.2771 0.3173 0.2744 0.2799 0.2705 0.2582 ant-1.7 0.2206 0.3865 0.2536 0.2415 0.2869 0.2744 0.2508 0.2464 0.2446 camel-1.0 0.0226 0.0226 0.0226 0.0226 0.0226 0.0197 0.0226 0.0505 0.0197 camel-1.2 0.6202 0.6368 0.614 0.6258 0.6777 0.6322 0.6297 0.6357 0.6435 camel-1.4 0.277 0.2881 0.2578 0.2585 0.2726 0.2825 0.274 0.2773 0.2814 camel-1.6 0.4125 0.3797 0.3822 0.3855 0.3607 0.3907 0.393 0.3865 0.3800 xerces-1.2 0.1509 0.0964 0.1505 0.1138 0.1055 0.1145 0.1164 0.1456 0.1176 xerces-1.3 0.345 0.2666 0.3003 0.3251 0.2985 0.289 0.4037 0.2963 0.3572 ivy-1.1 0.695 0.6248 0.6423 0.6259 0.6106 0.7889 0.6241 0.6104 0.5518 ivy-1.4 0.0424 0.033 0.0528 0.0424 0.0382 0.0372 0.0424 0.0659 0.0353 ivy-2.0 0.1028 0.0959 0.0909 0.1086 0.1038 0.0948 0.0931 0.1068 0.0913 mean 0.2594 0.2519 0.2499 0.2462 0.2481 0.2521 0.2467 0.2523 0.2390

[0062] Table 6

[0063] full Relief info chi2 HFSNFP FSCR FSDNP MIC-TSFS This method ant-1.3 0.487 0.6262 0.6209 0.6008 0.601 0.4717 0.6537 0.532 0.6515 ant-1.4 0.3973 0.2908 0.3488 0.4077 0.2131 0.5004 0.0 0.3701 0.2956 ant-1.5 0.5191 0.1976 0.3408 0.4227 0.274 0.2392 0.0986 0.3767 0.2339 ant-1.6 0.7539 0.7829 0.7686 0.771 0.7309 0.7699 0.7735 0.7793 0.7959 ant-1.7 0.7335 0.6765 0.7435 0.7356 0.7379 0.7483 0.7528 0.7375 0.7395 camel-1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0593 0.0000 camel-1.2 0.5484 0.4888 0.5425 0.5308 0.2917 0.4449 0.5257 0.529 0.5266 camel-1.4 0.5931 0.5847 0.6176 0.5793 0.6159 0.6367 0.5896 0.6 0.6194 camel-1.6 0.6262 0.587 0.6069 0.5931 0.5641 0.5586 0.5934 0.5984 0.6261 xerces-1.2 0.176 0.0 0.2805 0.1122 0.0 0.1157 0.1871 0.1885 0.1608 xerces-1.3 0.6352 0.7264 0.6239 0.5657 0.7591 0.6245 0.6479 0.6637 0.5760 ivy-1.1 0.5686 0.6149 0.583 0.472 0.6775 0.5189 0.4843 0.6379 0.5970 ivy-1.4 0.1685 0.0707 0.0707 0.0 0.0 0.0707 0.0 0.1186 0.1284 ivy-2.0 0.4094 0.3184 0.33 0.4507 0.3113 0.299 0.3858 0.2632 0.5804 mean 0.4726 0.4261 0.4627 0.4458 0.4364 0.4285 0.4066 0.4610 0.4665

[0064] As can be seen from Tables 4-6, on the linear regression model, the average AAE value of this method is 0.4390, the average ARE value is 0.2390, and the average G-Mean value is 0.4665. These values ​​outperform other methods on most datasets, indicating that the feature subset selected by this method has high predictive accuracy in the linear regression model.

[0065] Tables 7-9 show the AAE, ARE, and G-means results of 10-fold cross-validation on a decision tree regression model using the full feature dataset and feature subsets obtained by different feature selection methods:

[0066] Table 7

[0067] full Relief info chi2 HFSNFP FSCR FSDNP MIC-TSFS This method ant-1.3 0.5287 0.5117 0.4888 0.43 0.4016 0.4804 0.437 0.5119 0.3571 ant-1.4 0.4034 0.3496 0.3527 0.3846 0.3358 0.3167 0.4115 0.3568 0.2908 ant-1.5 0.1889 0.1825 0.1643 0.1634 0.1748 0.1585 0.1646 0.1651 0.1431 ant-1.6 0.5299 0.5468 0.5751 0.5457 0.6251 0.586 0.6187 0.513 0.5518 ant-1.7 0.5688 0.6876 0.5664 0.5714 0.6195 0.5791 0.6004 0.5634 0.5844 camel-1.0 0.0962 0.0681 0.0835 0.0897 0.0782 0.069 0.0919 0.105 0.0650 camel-1.2 1.0615 1.2546 1.0992 1.2114 1.2613 1.4195 1.1898 1.0999 1.0490 camel-1.4 0.5892 0.6041 0.6548 0.6359 0.4930 0.6247 0.6924 0.6191 0.5069 camel-1.6 0.8305 0.8383 0.9011 0.8797 0.8418 0.9557 0.9299 0.8995 0.6980 xerces-1.2 0.332 0.3609 0.3543 0.3455 0.3264 0.3691 0.4041 0.3457 0.3136 xerces-1.3 0.506 0.6381 0.4774 0.5236 0.5603 0.5206 0.4557 0.4836 0.4626 ivy-1.1 1.8976 1.7189 1.8608 1.8126 1.7124 1.8571 1.7206 2.0264 1.4864 ivy-1.4 0.1373 0.0975 0.1612 0.1323 0.1325 0.1429 0.144 0.1571 0.1243 ivy-2.0 0.2677 0.2282 0.2441 0.273 0.2538 0.2807 0.2569 0.2514 0.2101 mean 0.5622 0.5763 0.5749 0.5628 0.5578 0.5993 0.5796 0.5784 0.4888

[0068] Table 8

[0069] full Relief info chi2 HFSNFP FSCR FSDNP MIC-TSFS This method ant-1.3 0.3621 0.3617 0.3371 0.299 0.2429 0.3369 0.2918 0.3647 0.2554 ant-1.4 0.2985 0.2342 0.2533 0.2694 0.2295 0.2525 0.2948 0.245 0.1894 ant-1.5 0.1433 0.1331 0.1219 0.1135 0.1285 0.1202 0.1123 0.1253 0.0976 ant-1.6 0.2909 0.3095 0.3465 0.3155 0.3540 0.3583 0.3822 0.2695 0.3017 ant-1.7 0.3314 0.391 0.3326 0.3314 0.3731 0.3398 0.3458 0.3259 0.3415 camel-1.0 0.0757 0.0478 0.0626 0.0691 0.0581 0.0493 0.0709 0.0843 0.0448 camel-1.2 0.5981 0.7564 0.6243 0.7159 0.7634 0.877 0.6716 0.5989 0.5671 camel-1.4 0.3467 0.3525 0.4131 0.4083 0.2544 0.374 0.4477 0.3858 0.2859 camel-1.6 0.487 0.4738 0.5344 0.5016 0.4614 0.5548 0.5564 0.5034 0.3479 xerces-1.2 0.221 0.2443 0.2393 0.2115 0.2090 0.2515 0.2761 0.2324 0.1998 xerces-1.3 0.2721 0.3555 0.221 0.2421 0.3125 0.2606 0.2251 0.2447 0.2318 ivy-1.1 0.6257 0.6514 0.6799 0.7 0.7039 0.7103 0.672 0.7067 0.5549 ivy-1.4 0.1003 0.0572 0.1249 0.095 0.0932 0.1047 0.1058 0.1214 0.0860 ivy-2.0 0.1931 0.1562 0.1681 0.1879 0.1690 0.2049 0.1719 0.1777 0.1351 mean 0.3104 0.3227 0.3219 0.3099 0.3096 0.3437 0.3273 0.3133 0.2599

[0070] Table 9

[0071] full Relief info chi2 HFSNFP FSCR FSDNP MIC-TSFS This method ant-1.3 0.1766 0.3508 0.3203 0.3994 0.4265 0.3736 0.3611 0.481 0.6562 ant-1.4 0.5229 0.5614 0.6118 0.4504 0.4978 0.4712 0.4078 0.5309 0.4586 ant-1.5 0.4875 0.4142 0.5185 0.3615 0.4493 0.5788 0.2935 0.5527 0.4209 ant-1.6 0.6786 0.7248 0.6956 0.6668 0.6104 0.6624 0.6704 0.6926 0.6279 ant-1.7 0.6658 0.5288 0.6296 0.604 0.6041 0.5831 0.6191 0.6274 0.5552 camel-1.0 0.0661 0.0522 0.0295 0.0485 0.0420 0.0845 0.0277 0.0413 0.0707 camel-1.2 0.5756 0.5017 0.5558 0.5044 0.5045 0.4744 0.5296 0.5312 0.5811 camel-1.4 0.5437 0.4823 0.4683 0.5114 0.5310 0.4933 0.4798 0.5488 0.5247 camel-1.6 0.5472 0.5034 0.5011 0.4757 0.5010 0.4667 0.4664 0.5158 0.5376 xerces-1.2 0.642 0.6359 0.6153 0.503 0.6615 0.6196 0.5303 0.6269 0.5718 xerces-1.3 0.6614 0.5384 0.5577 0.551 0.6048 0.5265 0.5999 0.6052 0.6589 ivy-1.1 0.6185 0.6182 0.4946 0.5748 0.5209 0.657 0.5771 0.5068 0.6369 ivy-1.4 0.121 0.0 0.1544 0.1066 0.0340 0.0667 0.0856 0.1499 0.0535 ivy-2.0 0.3824 0.5717 0.2815 0.2877 0.2899 0.4487 0.326 0.3073 0.5526 mean 0.4823 0.4655 0.4545 0.4773 0.4481 0.4595 0.4314 0.4798 0.4934

[0072] As shown in the table above, our method outperforms existing methods in different baseline models, indicating that our method, which uses the optimization results of the attention mechanism as the evaluation index for feature selection, can indeed significantly improve the software defect prediction effect.

Claims

1. An integrated genetic feature selection method based on attention mechanism optimization, using genetic algorithm to select features of software defect dataset, and constructing feature subset, characterized in that: Two parent individuals are randomly selected and spliced into a two-dimensional parent matrix; the quality score of each gene in the parent matrix is calculated using the multi-head attention mechanism of the Transformer encoder, and the position of the cross is selected according to the quality score of the gene; the higher the quality score, the higher the probability of being selected; the offspring individual is generated through cross inheritance; After generating P offspring individuals, let the iteration number g = g + 1, and perform the next cross inheritance iteration until the termination condition of iteration is reached; The generated offspring individuals are used for feature selection of the software defect dataset to construct a feature subset; The feature subset is used to train the base model, the mean square error of the feature subset on the base model is calculated, and the negative value is taken as the fitness value; the feature subset with the highest fitness value is used to train the software defect prediction model.

2. The integrated genetic feature selection method based on attention mechanism optimization according to claim 1, wherein: Initialize the genetic algorithm, set the population size P and the termination iteration number G, initialize the iteration number g = 1; encode the genes of the individual using binary, 1 indicates that the feature is selected, 0 indicates that the feature is not selected, and the length of the individual is the same as the number of features in the software defect dataset.

3. The integrated genetic feature selection method based on attention mechanism optimization according to claim 2, wherein: The number of genes coded as 1 in the individual is fixed as k, which is used to control the number of features in the feature subset.

4. The integrated genetic feature selection method based on attention mechanism optimization of claim 3, wherein: The top k genes with the highest quality score are selected from the parent matrix as the genes coded as 1 in the offspring individual, and the remaining genes are coded as 0.

5. The integrated genetic feature selection method based on attention mechanism optimization of claim 1, wherein: The feature subset is used for 10-fold cross-validation of the base model, and the mean square error of the predicted results of the base model and the labeled defect number is calculated.

6. The integrated genetic feature selection method based on attention mechanism optimization according to claim 1 or 5, characterized in that: The feature subset is used to train different base models, the mean square error of the feature subset on each base model is calculated, the average value is calculated, and the negative value is taken as the fitness value of the corresponding feature subset.

7. The integrated genetic feature selection method based on attention mechanism optimization of claim 6, wherein: The base model includes a Bayesian regression model, a linear regression model, a decision tree regression model, and a support vector machine regression model.

8. The integrated genetic feature selection method based on attention mechanism optimization of claim 1, wherein: The position encoding enhancement is performed on each row of the two-dimensional parent matrix, and then the quality score of each gene in the parent matrix is calculated using the multi-head attention mechanism of the Transformer encoder.

9. A software defect prediction method, characterized by: Features and corresponding defect conditions of multiple software modules are obtained from a software development project as an original dataset; the original dataset is selected using the method of any of claims 1-5 or 8 to construct a training subset; The training subset is used to train the software defect prediction model, and the features of the software module are input into the trained model to predict the number of defects in the software module.

10. The software defect prediction method of claim 9, wherein: The software defect prediction model is one of a Bayesian ridge regression optimized model, a linear regression model, or a decision tree regression model.

Images