A software defect prediction model based on feature mapping and attribute compensation technology
Through transfer learning and attribute compensation technology, the problem of data distribution differences in cross-project defect prediction is solved, the similarity matching between source and target data is achieved, and the performance and robustness of the defect prediction model are improved.
Patent Information
- Application Number
- CN202110851716.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-07-27
- Publication Date
- 2024-12-20
- Estimated Expiration
- 2041-07-27
AI Technical Summary
In software engineering, the lack of historical data of new development projects, which makes it difficult to meet the independent and homogeneous distribution hypothesis in cross-project defect prediction. Traditional methods cannot effectively build a high-performance defect prediction model.
Combining transfer learning technology and attribute compensation technology, the data distribution of source and target projects is made more similar through cross-validation, sampling and normalized preprocessing, metric compensation and semi-supervised migration component analysis (SSTCA) methods, and the decision tree classification model is used. Make defect predictions.
It improves the robustness and prediction performance of the cross-project defect prediction model, enhances the data distribution similarity between the source and target data domains, and improves the generalization ability and comprehensiveness of the model.
Smart Images
Figure CN113626316B_ABST
Abstract
Description
Technical Field
[0001] The invention belongs to the field of software security and relates to a software project defect prediction model based on feature mapping and metric compensation method. Background Art
[0002] With the vigorous development of Internet technology and the increasing complexity of software, software security and software reliability are becoming increasingly important in software engineering. In order to ensure the high reliability of software quality, defect prediction methods have become a research hotspot in the field of defect prediction. Good software defect prediction methods can greatly improve the testing efficiency of software. Therefore, building a high-performance defect prediction model is of great significance in ensuring software quality.
[0003] Software defect prediction is based on the historical data accumulated during the software development process to build a prediction model. However, when there is no historical data or the historical defect data is insufficient to build a model, the traditional software defect prediction method based on the target data cannot meet the practical needs. In actual applications, the projects to be predicted are usually newly developed projects, which leads to the problem of insufficient historical data mentioned above. This is where the concept of cross-project defect prediction comes in, that is, using the data of other related projects as training to build a defect prediction model for the target data. This leads to the problem of data differences between the source project and the target project. Different projects have different feature spaces and distributions of software defect data due to their different contexts, that is, the difference in the distribution of metric values of the source and target project data sets makes it difficult to meet the assumption of independent and identical distribution. Using traditional machine learning techniques to build defect prediction models cannot achieve better prediction results.
[0004] The current cross-project defect prediction methods can be roughly divided into two categories: attribute conversion-based and data selection-based. The attribute feature transformation makes the source project and the target project obey the same distribution while retaining their respective defect features. In order to solve the problem that the data distribution differences between different projects are large, resulting in low performance of the defect prediction model, we use the metric compensation method to change the data distribution in the source data set to adapt to the target data set, and use the idea of weights during instance training to reduce the distribution difference between the source and target project data. Then, the transfer learning technology uses the existing knowledge of the source project to solve the problem that there are only a few or even no labeled instances in the target field. The semi-supervised transfer component analysis (SSTCA) method is used to obtain the minimum data distance between the source project and the target project in the mapping space, while retaining their respective internal attributes to the greatest extent. The effects of these two methods are combined to enhance the similarity of the data distribution of the source and target data domains. That is, the idea of feature mapping and attribute value transformation is used to make the data distribution in the source and target data sets in cross-project defect prediction as similar as possible. Summary of the invention
[0005] The existing methods for cross-project defect prediction problems can be roughly divided into two categories, one is based on attribute conversion, and the other is based on data selection. Among them, the methods based on attribute conversion mostly use the idea of weight to make the source and target data more consistent and obtain similar data distribution. The present invention combines a transfer learning method SSTCA that can fully utilize the class label information of the source domain data for transfer component learning with attribute compensation technology, so that the data distribution of the target project is similar to that of the source project. SSTCA makes full use of the class label information of the data in the source project, uses the transfer data of the source project as training data, and uses sampling methods to further perform class imbalance learning. At the same time, the mean of the prediction results of the multi-source training set for the same target data set is used as the final prediction result of the model. The experimental results of this method are compared with the existing metric compensation method to verify the high efficiency of the method of the present invention.
[0006] The present invention proposes a cross-project defect prediction model that combines transfer learning technology and attribute compensation technology, including:
[0007] Step 1: Use cross-validation to reasonably divide the data set, and perform sampling and normalization preprocessing on the sample set used for testing to obtain a more balanced defect data set;
[0008] Step 2: Apply attribute compensation method to solve the problem of data distribution differences across projects, so that the data distribution of the target project is more consistent with the data distribution of the source project;
[0009] Step 3, after obtaining the sample set processed by the metric compensation technology, the transfer learning technology is used to minimize the data distance between the source and target data to obtain a more similar data distribution;
[0010] Step 4: Based on the training sample set and test sample set obtained in step 3, the decision tree classification model is used to classify the defects of the data, and the prediction performance of the model is evaluated based on the prediction results.
[0011] In the first aspect, the specific steps of the above step 1 are as follows:
[0012] Step 1.1, use the loadtxt() file reading method to reasonably divide the data set required for model performance verification, read out the measurement data and label data in the corresponding training sample set and test sample set, use "," as the reading segmentation mark, read the first N columns of the data as measurement data and store them in the x_list list, and the data in the N+1th column is the label of the defect data and is stored in the y_list list;
[0013] Step 1.2, use the deviation normalization method to normalize the numbers after the standardized operation, convert the standardized data to [0,1], and obtain the normalized training set data, so that the data of each metric attribute has a stronger visualization operation.
[0014] Step 1.3: Use the SMOTE sampling method to oversample the training data, effectively alleviating the class imbalance problem of defective data and improving the training accuracy of the data set. Set the sampling ratio according to the sample imbalance ratio to determine the sampling rate. For each sample in the minority class, calculate its distance to all samples in the minority class sample set based on the Euclidean distance. After sorting, obtain the k nearest neighbors of the sample and then expand the minority class samples.
[0015] In the second aspect, the calculation process of the above attribute compensation method includes:
[0016] Step 2.1, store the preprocessed training data sample set in the form of a list, and for each sample in the target training set, multiply the original data by a weight value, which is the ratio of the mean of the training data to the target data, so as to obtain a new target data set that is more suitable for the distribution of the source data. Similarly, for all samples in the source data set, use the ratio of the mean of the target data to the source data as the new weight assigned by the source data set to adapt to the target data, so as to obtain a data set that adapts to both the source and target data after a round of attribute value conversion. The calculation method for the above source and target data conversion is:
[0017]
[0018] Where: m
[0019] In the above formula, source[i,j] represents the jth metric value of the i-th data instance in the source project data, and metric_mean_source represents the mean of all data instances in the source project in the j-th metric. Similarly, target[i,j] represents the j-th metric value of the i-th data instance in the target project data, and metric_mean_target represents the mean of all data instances in the target project in the j-th metric.
[0020] In step 2.2, sour1[i,j] is used to represent the jth metric value of the i-th data instance of the required new source data obtained by step 2.1. Metric_mean_source and metric_mean_target are used to represent the mean of the j-th metric of the source data and the target data after recalculation. The source data is processed again in a similar way to step 2.1, so that the source data set is converted into a secondary attribute value according to the distribution of the target data, thereby further improving the similarity and fit between the source data and the target data, and completing the second round of metric mapping between the new source and target data. The calculation method of sour1 is:
[0021] sour1[i,j]=(sour1[i,j]*metric_mean_target) / metric_mean_source.
[0022] In step 2.3, the source data and target data after attribute value transformation are passed in as parameters of the new transfer learning method, so as to carry out the next step of feature mapping-based transformation.
[0023] Thirdly, the computational process of using the semi-supervised transfer component analysis method to process the distribution of source and target data is as follows:
[0024] Step 3.1: Use SSTCA migration component analysis technique to further improve the training data (X S ) and test data (X T ) distribution, thereby improving the robustness of the defect prediction model, and the distance between the two domains is calculated using the MMD (maximum mean difference) algorithm, where MMD (X S , X T )=tr(KL), L is the matrix introduced by the MMD algorithm, K is the kernel matrix obtained by mapping with the kernel function, and tr(KL) represents the trace of the matrix obtained by solving the concatenated matrix KL.
[0025] Step 3.2, using the kernel function to transform the quantized distance into the kernel function learning process, first calculate the custom L and H matrices, where the definition Here H is a central matrix, n1, n2 are the source domain X src and the target domain X tar The number of instances, x i , x j Represents the sample data in the domain, and then calculates the kernel matrix K.
[0026] Step 3.3, use the kernel matrix K calculated in step 3.2 to solve (KLK+μI) -1KHK first m eigenvalues, as mentioned above, L is the matrix introduced by the MMD algorithm, K is the kernel matrix obtained by mapping the kernel function, H is the center matrix, μ is the introduced parameter, and I is the intermediate matrix introduced in the algorithm. The SSTCA algorithm is used to calculate the matrix after dimensionality reduction and pass it into the source domain X S and the target domain X T , and the desired data dimension dim after dimensionality reduction, the new source data features and new target data features (i.e. the result of dimensionality reduction of the source domain and the data domain). The number of rows of X is the number of features of the original data, and the number of columns is the total number of features.
[0027] In the fourth aspect, the specific steps of the above step 4 are as follows:
[0028] Step 4.1: For the training sample set, after the above-mentioned attribute conversion and migration component analysis operations, defect data with similar distribution and reduced dimension are obtained, and the corresponding decision tree classifier object is created. The decision tree classifier is used to predict the category of defect data for the target data set.
[0029] In step 4.2, the prediction result for the same target data is comprehensively determined by the models trained with multiple different training data. For a selected target data, several classification models with different prediction performances are trained using multiple different training data sets. Finally, the average of the prediction results of multiple models is used as the final prediction value of the target data.
[0030] Compared with the existing methods, the present invention has the following beneficial effects:
[0031] 1. A cross-project defect prediction model that combines transfer learning technology and metric compensation technology is proposed. It predicts defect data based on similar source and target data distributions. The attribute compensation technology is used to allow the source data to adapt to the distribution of the target data using the idea of weights during training. After obtaining relatively suitable data, a secondary metric value conversion is performed again, which takes into account the distribution of the source and target data at the same time, thereby making the constructed defect prediction model more robust.
[0032] 2. The advantage of processing data distribution based on transfer learning technology is that when building a defect prediction model, the inconsistent distribution of training data and test data will lead to poor prediction performance of the trained model, and it may be difficult to predict the correct result in the test. In order to make the data distribution of the two domains more similar, we use the maximum mean difference MMD algorithm to quantify the distribution distance of the two domains. The essence of TCA is to minimize MMD, so that the difference between samples in the two domains is kept to a minimum. This further enhances the similarity between different data domains. BRIEF DESCRIPTION OF THE DRAWINGS
[0033] Figure 1It is an overall flow chart of a cross-project defect prediction method that combines feature mapping and attribute value conversion techniques.
[0034] Figure 2 It is a detailed flow chart of a cross-project defect prediction method combining feature mapping and attribute value conversion techniques.
[0035] Figure 3 It is the data sample set information used in the experimental link of the present invention.
[0036] Figure 4 It is the information and results of experiments on defect prediction models obtained by data processing using different schemes, with KC2 as the target project and CM1, KC1, JM1, and PC1 as the source project training sets.
[0037] Figure 5 It is the information and results of experiments on defect prediction models obtained by data processing using different schemes, with CM1 as the target project and KC2, KC1, JM1, and PC1 as the source project training set.
[0038] Figure 6 It is the information and results of experiments on defect prediction models obtained by data processing using different schemes, with KC1 as the target project and CM1, KC2, JM1, and PC1 as the source project training sets.
[0039] Figure 7 It is the information and results of experiments on defect prediction models obtained by data processing using different schemes, with PC1 as the target project and CM1, KC1, JM1, and KC2 as the source project training set.
[0040] Figure 8 The AUC value, recall and precision of the experiment are obtained by using KC2 as the target project and CM1, KC1, JM1 and KC2 as the source project training sets and using different schemes for data processing.
[0041] Fig. 9 The AUC value, recall and precision of the experiment are obtained by taking PC1 as the target project and KC1, CM1, JM1 and KC2 as the source project training sets and using different schemes for data processing.
[0042] Fig.10 The AUC value, recall and precision of the experiment are obtained by taking KC1 as the target project and PC1, CM1, JM1 and KC2 as the source project training sets and using different schemes for data processing. DETAILED DESCRIPTION
[0043] The present invention is further described below in conjunction with the accompanying drawings and implementation examples. It should be noted that the implementation examples described are only intended to facilitate the understanding of the present invention and do not have any limiting effect on it.
[0044] The present invention aims to address the problem that there is a large difference in data distribution between the source data set and the target data set when constructing a defect prediction model in cross-project defect prediction. A method for improving the similarity between source and target data is proposed to establish a cross-project defect prediction model with better prediction performance and higher robustness. Sufficient experiments are carried out to prove the feasibility and efficiency of the method.
[0045] like Figure 1 As shown, a cross-project defect prediction method combining transfer learning and attribute compensation technology of the present invention includes:
[0046] Step 201 uses the loadtxt() file reading method to reasonably divide the data set required for model performance verification, reads out the metric data and label data in the corresponding training sample set and test sample set, uses "," as the reading segmentation mark, and stores the first N columns of the read data as metric data in x_list, and the data in the N+1th column is the label of the defect data in y_list; the sample set used for the test is sampled and normalized to obtain a more balanced defect data set;
[0047] The purpose of implementing data preprocessing in the present invention is that when the source data set is used as training data, each project data set is composed of many sample data, and each defect sample number is represented by several measurement data, but the measurement dimensions are different between different measurements. Therefore, when training the defect prediction model, it is necessary to perform preprocessing operations on the training data samples so that different measurements have the same scale. In short, when the measurement units of the training data in different dimensions are inconsistent, standardized steps are required to preprocess the data.
[0048] Step 2011 uses the deviation standardization method to normalize the numbers that have undergone the standardization operation, and converts the standardized data into [0,1], so that data with different measurement attributes can have stronger visualization operations.
[0049] The normalization calculation process includes: (1) calculating the maximum and minimum values in the sample data, (2) using the conversion function Process the measurement values and label values in the data to convert the data samples to [0,1], f / This is the normalized data.
[0050] Step 2012 uses the SMOTE oversampling method to sample and expand the training data. The sampling ratio is set according to the sample imbalance ratio to determine the sampling rate. For each sample in the minority class, the distance from it to all samples in the minority class sample set is calculated based on the Euclidean distance standard, and the k-nearest neighbor expanded minority class samples are obtained after sorting.
[0051] The sampling calculation process includes: for each randomly selected neighbor x n ,according to By constructing new samples based on the formula, a more balanced data set can be obtained according to the set sampling ratio.
[0052] Through the above operations, the preprocessed standard input data can be obtained.
[0053] Step 202, applying an attribute compensation method to solve the problem of cross-project category balance, so that the data distribution of the target project is more consistent with the data distribution of the original project;
[0054] In the above step 202, the steps of processing data distribution by the attribute compensation method are as follows:
[0055] Step 2021 stores the preprocessed training data sample set in the form of a list. For each sample in the target training set, a weight value is assigned to the original data. The weight value is the ratio of the training data to the mean of the target data, thereby obtaining a new target data set that is more suitable for the distribution of the source data. Similarly, for all samples in the source data set, the ratio of the target data to the mean of the source data is used as the new weight for processing the source data set, thereby obtaining a data set that is suitable for the source data and the target data after a round of attribute value conversion. The calculation method for the above source and target data conversion is:
[0056] source[i,j]=(source[i,j]*metric_mean_target) / metric_mean_source
[0057] target[i,j]=(target[i,j]*metric_mean_source) / metric_mean_target
[0058] in:
[0059] metric_mean_source=np.mean(source[:,j])
[0060] metric_mean_source=np.mean(target[:,j])
[0061] In the above formula, source[i,j] represents the jth metric value of the i-th data instance in the source project data, and metric_mean_source represents the mean of all data instances in the source project in the j-th metric. Similarly, target[i,j] represents the j-th metric value of the i-th data instance in the target project data, and metric_mean_target represents the mean of all data instances in the target project in the j-th metric.
[0062] In step 2022, sour1[i,j] is used to represent the jth metric value of the i-th data instance of the required new source data obtained by step 2021. Metric_mean_source and metric_mean_target are used to represent the mean of the jth metric of the source data and the target data after the processing. Using the idea of weight, the source data set is transformed into a secondary attribute value according to the distribution of the target data, which further improves the similarity and fit between the source and target data, that is, the second round of metric mapping between the new source and target data is completed. The calculation method of sour1 is:
[0063] sour1[i,j]=(sour1[i,j]*metric_mean_target) / metric_mean_source.
[0064] Step 203, after obtaining the sample set processed by the metric compensation technology, the transfer learning technology TCA is used to minimize the data distance between the source and target data to obtain a more similar data distribution;
[0065] In the above step 203, the steps of using the SSTCA technology to change the distribution of source and target data are as follows:
[0066] Step 2031 uses the source and target data obtained in step 202 after two rounds of mapping as the parameters of the SSTCA algorithm, and uses migration component analysis to perform another transformation on the data mapping.
[0067] Step 2032, convert the difference between the two domains into the distance between the two domains for calculation, and use the MMD (maximum mean difference) algorithm to quantitatively measure. The learning process of using the kernel function to convert the quantized distance into the kernel function first calculates the custom L and H matrices, where the definition Here H is a central matrix, n1, n2 are the source domain X src and the target domain X tar The number of instances, x i , x j Represents the sample data in the domain, and then calculates the kernel matrix K.
[0068] Step 2033, using the kernel matrix K calculated in step 2032, to solve (KLK+μI) -1 KHK is the first m eigenvalues. As mentioned above, L is the matrix introduced by the MMD algorithm, K is the kernel matrix obtained by mapping the kernel function, H is the center matrix, μ is the introduced parameter, and I is the intermediate matrix introduced in the algorithm. The SSTCA algorithm calculates the matrix after dimensionality reduction, and passes in the source domain Xs and the target domain Xt, as well as the desired data dimension dim after dimensionality reduction. The new source data features and new target data features obtained are the results of dimensionality reduction of the source domain and the data domain. The number of rows in X is the number of features of the original data, and the number of columns is the total number of features.
[0069] Step 204, based on the training sample set and the test sample set obtained in step 203, the decision tree technology is used to classify the defects of the data to obtain the defect prediction performance of the model.
[0070] Step 2041: After the attribute conversion and migration component analysis operations are performed on the training sample set, defect data with similar distribution and reduced dimension are obtained, and a corresponding decision tree classifier object is created. The decision tree classifier is used to predict the category of defect data for the target data set;
[0071] In step 2042, the prediction result for the same target data is comprehensively determined by the models trained with multiple different training data. For a selected target data, several classification models with different prediction performances are trained using multiple different training data sets. Finally, the average of the prediction results of multiple models is used as the final prediction value of the target data.
[0072] This paper mainly proposes a method to solve the problem of data distribution difference in source domain target data in cross-project defect prediction. We combine transfer learning and attribute compensation technology to process data and convert attribute values. We select 5 subsets of NASA data set as experimental data, namely CM1, JM1, KC1, KC2, PC1, Figure 3 Shows the details of the dataset used in the experiments.
[0073] In order to improve the performance of the cross-project defect prediction model, the present invention proposes some solutions to solve the data differences between different projects. Since the distribution of metric data of different projects is very different, the present invention uses the idea of weight to perform attribute conversion operations on the source and target data to minimize the data differences between projects. At the same time, the transfer component analysis technology SSTCA is combined to perform secondary mapping of the attributes. When predicting the target data, the prediction results of multiple classifiers with different performance are taken as the final prediction value of the model, which greatly improves the generalization ability and prediction performance of the model.
[0074] pass Figure 8-Figure 10 The line graph can be seen intuitively. For the four data processing methods, for the five selected data sets, the method proposed in 2017 of using the target data set to adapt to the distribution of the source data set is better than the method of using the source data set to adapt to the distribution of the target data set in 2008 in terms of recall rate index, which shows that this method can predict more comprehensive defect data for the model. Therefore, this study also uses the idea of using the target data to adapt to the distribution of the source data, and on this basis, combines the SSTCA dimensionality reduction method based on feature mapping to keep the test data and the target data attribute values with the greatest degree of data similarity, thereby improving the prediction effect of the defect prediction model, so that the model prediction effect is optimal.
[0075] After determining the idea of using target data to adapt to the distribution of source data, the present invention studies how to design appropriate weight values to make the source data set more adaptable to the target data. For the attribute compensation method, we make the target data adapt to the distribution of the source data set, and also make the source data set undergo a secondary conversion to adapt to the target data. This paper conducts 500 experiments on each source and target data for each method, and finally takes the average of the experimental results of different data sets as the final performance of the model. Figure 4-Figure 7 The recall rate, AUC value and precision rate of the data distribution processed by combining SSTCA feature mapping and attribute value conversion are shown. At the same time, as a comparison, this paper also conducted similar experiments on the attribute value conversion methods used in 2008 and 2017.
[0076] After the selection of the data processing method, the defect data of the five data sets selected in the experiment are predicted. The method proposed in the present invention is used to deal with the distribution difference problem of the source and target data, and then the source and target data with similar data distribution are obtained. Finally, the classifier trained with the transfer data as training data is used to complete the prediction of the target data. In addition, in order to verify the effectiveness of the method proposed in the present invention, multiple groups of comparative experiments are set up. Experiments are conducted on the same test and target data sets using different data processing methods, and comprehensive comparisons are made from indicators such as AUC. The experimental results are as follows: Figure 4-Figure 7 shown.
[0077] By observation Figure 8-Figure 10It can be seen that the algorithm proposed in the present invention can enable the decision tree classifier to obtain a more accurate defect prediction effect, which is because the method combines the data distribution of the source and target data to train the model. On the other hand, the method uses the class label information of the data in the source project through the migration component analysis technology, minimizes the data distance between the source and target data in the mapping space, and uses the transfer data of the source project as training data, that is, through attribute value conversion and feature mapping-based transfer learning technology, the distribution similarity of the source and target data is guaranteed to the greatest extent, thereby greatly improving the prediction effect of the defect prediction system. It can be seen that the method proposed in this study can effectively improve the method based on metric value conversion, and the improved method can improve the performance of the prediction model to a certain extent.
Claims
1. A software defect prediction method based on feature mapping and attribute conversion, characterized in that: The steps include: Step 1: Use cross-validation to reasonably divide the data set, and perform sampling and normalization preprocessing on the sample set used for the test to obtain a balanced defect data set; Step 2: Apply attribute compensation method to solve the problem of data distribution differences across projects, so that the data distribution of the target project matches the data distribution of the source project; Step 3, after obtaining the sample set processed by the metric compensation technology, the transfer learning technology is used to minimize the data distance between the source data and the target data to obtain a similar data distribution; Step 4: Based on the training sample set and the test sample set obtained in step 3, the decision tree classification model is used to classify the defects of the data, and the prediction performance of the model is evaluated based on the prediction results; The specific implementation of step 2 includes the following steps: Step 2.1, store the preprocessed training data sample set in the form of a list. For each sample in the target training set, multiply the original data by a weight value, which is the ratio of the mean of the training data to the target data, so as to obtain a new target data set that is more suitable for the distribution of the source data. Similarly, for all samples in the source data set, use the ratio of the mean of the target data to the source data as the new weight assigned by the source data set to adapt to the target data, so as to obtain a data set that adapts to both the source and target data after a round of attribute value conversion. The calculation method of the above source and target data conversion is: in: Where source[i,j] represents the jth metric value of the i-th data instance in the source project data, and metric_mean_source represents the mean of all data instances in the source project in the j-th metric; similarly, target[i,j] represents the j-th metric value of the i-th data instance in the target project data, and metric_mean_target represents the mean of all data instances in the target project in the j-th metric; In step 2.2, sour1[i,j] is used to represent the jth metric value of the i-th data instance of the required new source data obtained by step 2.
1. Metric_mean_source and metric_mean_target are used to represent the mean of the j-th metric of the source data and the target data after recalculation. The source data is processed again as in step 2.1, so that the source data set is converted into a secondary attribute value according to the distribution of the target data, thereby further improving the similarity and fit between the source data and the target data, and completing the second round of metric mapping between the new source and target data. The calculation method of sour1 is: sour1[i,j]=(sour1[i,j]*metric_mean_target) / metric_mean_source; Step 2.3, the source data and target data after attribute value transformation are passed in as parameters of the new transfer learning method, so as to carry out the next step of feature mapping-based transformation; The specific implementation of step 3 includes the following steps: Step 3.1: Use SSTCA migration component analysis technique to further improve the training data X S And test data X T The distribution of the defect prediction model is improved, and the distance between the two domains is calculated using the maximum mean difference MMD algorithm, where MMD(X S , X T )=tr(KL), L is the matrix introduced by the MMD algorithm, K is the kernel matrix obtained by mapping using the kernel function, and tr(KL) represents the trace of the matrix obtained by solving the concatenated matrix KL; Step 3.2, using the kernel function to transform the quantized distance into the kernel function learning process, first calculate the custom L and H matrices, where the definition Here H is a central matrix, n1, n2 are the source domain X src and the target domain X tar The number of instances, x i , x j Represents the sample data in the domain, and then calculates and obtains the kernel matrix K; Step 3.3, use the kernel matrix K calculated in step 3.2 to solve (KLK+μI) -1 KHK first m eigenvalues, as mentioned above, L is the matrix introduced by the MMD algorithm, K is the kernel matrix obtained by mapping the kernel function, H is the center matrix, μ is the introduced parameter, and I is the intermediate matrix introduced in the algorithm; the matrix after dimensionality reduction is calculated using the SSTCA algorithm and passed into the source domain X S and the target domain X T , and the desired data dimension dim after dimensionality reduction, the new source data features and the new target data features obtained, that is, the results of dimensionality reduction of the source domain and the data domain, where the number of rows of X is the number of features of the original data, and the number of columns is the total number of features.
2. The method according to claim 1, characterized in that: The specific implementation of step 1 includes the following steps: Step 1.1, use the loadtxt() file reading method to reasonably divide the data set required for model performance verification, read out the measurement data and label data in the corresponding training sample set and test sample set, use "," as the reading segmentation mark, read the first N columns of the data as measurement data and store them in the x_list list, and the data in the N+1th column is the label of the defect data and is stored in the y_list list; Step 1.2, use the deviation standardization method to normalize the numbers after the standardization operation, convert the standardized data to [0,1], and obtain the standardized training set data, so as to realize the visualization operation of the data of each metric attribute; Step 1.3: Use the SMOTE sampling method to oversample the training data to alleviate the class imbalance problem of defective data and improve the training accuracy of the data set. Set the sampling ratio according to the sample imbalance ratio to determine the sampling rate. For each sample in the minority class, calculate its distance to all samples in the minority class sample set based on the Euclidean distance. After sorting, obtain the k nearest neighbors of the sample and then expand the minority class samples.
3. The method according to claim 1, characterized in that: The specific implementation of step 4 includes the following steps: Step 4.1: For the training sample set, after the above attribute conversion and migration component analysis operations, defect data with similar distribution and reduced dimension are obtained, and the corresponding decision tree classifier object is created, and the decision tree classifier is used to predict the category of defect data for the target data set; In step 4.2, the prediction result for the same target data is comprehensively determined by the models trained with multiple different training data. For a selected target data, several classification models with different prediction performances are trained using multiple different training data sets. Finally, the average of the prediction results of multiple models is used as the final prediction value of the target data.