Feature selection stability improvement method for performance prediction and ensemble learning in production processes

Abstract

The present invention discloses a method for improving the stability of feature selection for performance prediction and integrated learning of production processes. It includes collecting real-time data; using statistical methods to divide the pre-processed data according to different production conditions, and using orthogonal test methods to classify the divided data sets to obtain classified data sets; performing multicollinearity diagnosis on the features in each classified data set to obtain feature independent data sets; for each feature independent data set, using different feature selection algorithms to perform feature selection to obtain feature subsets, and evaluating the performance weights of different feature selection algorithms through all feature subsets; re-ranking the feature set according to the weighted score of each feature to obtain a preliminary feature ranking; calculating the weights of ignored important features to obtain the optimal feature subset. The present invention has a higher probability of selecting a stable optimal feature subset, and can screen out a stable feature subset with high prediction performance.

Description

Technical Field

[0001] The present invention relates to the technical field of data mining, and in particular to a method for improving the stability of feature selection for performance prediction and integrated learning of a production process. Background Art

[0002] With the development of big data, the amount of data in various industries is showing a trend of large-scale growth. Data mining faces problems such as large data volume and high feature dimension. These problems will interfere with the accuracy of data analysis, thereby increasing the computational cost and time complexity. Therefore, data preprocessing is required when analyzing data samples. Feature selection is an important step in data preprocessing. It can not only help the model filter out redundant and irrelevant features, reduce data dimensions, and improve the computational efficiency and generalization ability of the model; but also better understand the process described by the data through the optimal feature subset. Therefore, it is particularly important to perform feature selection before these complex data enter the data-driven model.

[0003] With the increasing proliferation of high-dimensional data, feature selection algorithms have become an indispensable part of the learning process. In the prior art, a lot of work has been done on the stability of learning algorithms, but the stability of feature selection algorithms has been ignored. Stable feature selection technology is an important research issue. When the optimal feature subset is selected, a simpler, faster, and more accurate prediction model can be constructed.

[0004] However, the feature selection in the prior art has the following problems:

[0005] First, the data itself has characteristics such as uncertainty, strong coupling, and complex nonlinearity, which makes the feature selection algorithm prone to falling into local optimality;

[0006] Second, when performing feature selection on different batches of data with the same features, different optimal feature selection results may be produced, resulting in unstable feature selection. Summary of the invention

[0007] In view of the shortcomings of the prior art, the present invention proposes a method for improving the stability of feature selection for performance prediction and ensemble learning of production processes. Firstly, according to the distribution characteristics of the data set in different periods and working conditions, while ensuring data diversity, the influence of different training data on the results of the feature selection algorithm is reduced; secondly, from the three performance evaluation criteria of stability judgment criterion, prediction performance judgment criterion, and computational complexity judgment criterion, the influence of the performance weight of the feature selection algorithm on the results of the aggregation process is reduced, and the stability of feature selection and the probability of obtaining the optimal feature subset are increased; finally, the ignored important features are weighted, and higher weights are given to the features of important areas, and lower weights are given to the features of unimportant areas, so as to ensure that important features are selected.

[0008] To achieve the above-mentioned purpose, the present invention is designed to improve the stability of feature selection for performance prediction and ensemble learning of production processes, which is particularly characterized by comprising the following steps:

[0009] S1) Collecting real-time data of the production process and preprocessing the real-time data;

[0010] Real-time data collection is done by reading sensors at each production line location to obtain relevant production data;

[0011] The preprocessing includes filling, deleting, and data normalization. The data normalization refers to removing the unit restrictions of the data and converting the data into dimensionless pure values ​​to facilitate comparison and weighting of indicators of different units or magnitudes. The data normalization adopts the Z-score normalization method. The specific formula is as follows:

[0012]

[0013] In the formula,

[0014] x is the initial value of all sample data,

[0015] u is the mean of all sample data,

[0016] σ is the standard deviation of all sample data,

[0017] is the normalized value of all sample data;

[0018] S2) using statistical methods to divide the preprocessed data according to different production conditions to obtain data sets under different production conditions, and using orthogonal test methods to classify the divided data sets to ensure the average distribution of each influencing factor and each level of the orthogonal test, achieve uniform dispersion of the influencing factors and levels, and obtain classified data sets;

[0019] In S2), the influencing factors in the orthogonal experiment include time period, working conditions and varieties, and each influencing factor has 4 levels;

[0020] S3) performing multicollinearity diagnosis on the features in each classification data set, setting a correlation coefficient threshold, if the feature correlation coefficient is greater than the correlation coefficient threshold, it is determined that the feature multicollinearity is strong, if the feature correlation coefficient is less than the correlation coefficient threshold, it is determined that the feature multicollinearity is weak; according to the multicollinearity diagnosis result, the features with weak multicollinearity are divided into independent feature data sets, and the features with strong multicollinearity are divided into several non-independent feature data sets; for each non-independent feature data set, only one feature with the highest correlation with the target feature is retained as an independent feature; several independent features are obtained through several non-independent feature data sets, and the several independent features are added to the independent feature data set to obtain feature independent data sets;

[0021] S4) repeating step S3) to process all classification data sets into feature-independent data sets;

[0022] S5) for each feature independent data set, using different feature selection algorithms to perform feature selection, obtain feature subsets, and evaluate the performance weights of different feature selection algorithms through all feature subsets;

[0023] The specific evaluation steps of the performance weights of different feature selection algorithms are as follows:

[0024] S51) calculating the stability of each feature selection algorithm on each feature independent data set, and taking the stability mean as the stability evaluation criterion of the feature selection algorithm;

[0025] The stability of each feature selection algorithm on different feature independent data sets is obtained by the following formula

[0026]

[0027] In the formula,

[0028] S R (r, r′) represents the stability coefficient of each feature selection algorithm on the feature independent data sets D and D′, which is used to measure the stability of feature selection. R (r, r′) = [-1, 1],

[0029] r represents the result of running the same feature selection algorithm on the feature-independent data set D,

[0030] r′ represents the result of running the same feature selection algorithm on the feature-independent data set D',

[0031] r i represents the ranking of the running results of the i-th feature selection algorithm on the feature-independent data set D,

[0032] r′ irepresents the ranking of the results of the i-th feature selection algorithm on the feature-independent dataset D',

[0033] m represents the number of feature subsets,

[0034] i represents the number of feature selection algorithms;

[0035] The stability mean is obtained by the following formula

[0036]

[0037] In the formula,

[0038] Sr represents the stability mean of each feature selection algorithm,

[0039] S R (r, r′) represents the stability coefficient of the same feature selection algorithm on feature-independent data sets D and D′, which is used to measure the stability of feature selection.

[0040] r represents the result of running the same feature selection algorithm on the feature-independent data set D,

[0041] m represents the number of feature subsets,

[0042] m×(m-1) / 2 means the number of similar results obtained by comparing m feature subsets pairwise;

[0043] S52) constructing a prediction model based on data-driven, inputting the feature subsets selected by each feature selection algorithm on different feature independent data sets into the prediction model for prediction, and taking the mean of the prediction accuracy as the prediction performance evaluation criterion of the feature selection algorithm;

[0044] The mean prediction accuracy of each feature selection algorithm is obtained by

[0045]

[0046] In the formula,

[0047] MAE represents the mean prediction accuracy of each feature selection algorithm.

[0048] N l Indicates the number of values,

[0049] L represents the actual value,

[0050] L′ represents the predicted value,

[0051] |LL′| represents the prediction error value;

[0052] S53) calculating the time consumed by each feature selection algorithm in the feature selection process, and taking the average of the consumed time as the computational complexity evaluation criterion of the feature selection algorithm;

[0053] S54) calculating a stability weight according to the mean stability of each feature selection algorithm, calculating an accuracy weight according to the mean prediction accuracy of each feature selection algorithm, calculating a computational complexity weight according to the mean consumption time of each feature selection algorithm, and calculating a performance weight of each feature selection algorithm by combining the stability weight, the accuracy weight, and the computational complexity weight;

[0054] S6) sorting and scoring the positions of the k A features selected from all feature subsets in all feature selection algorithms, and then adding the k sorting scores corresponding to the k A features by the same feature selection algorithm to obtain the final score of the A feature, and using the final score of the A feature as the integrated feature sorting result corresponding to the A feature by the feature selection algorithm, and finally multiplying the integrated feature sorting result corresponding to the A feature by each feature selection algorithm by the performance weight of the corresponding feature selection algorithm and adding them to obtain the weighted score of the A feature; re-sorting the feature set according to the weighted score of each feature to obtain a preliminary feature sorting;

[0055] The preliminary ranking result of the features is obtained by the following formula

[0056]

[0057] In the formula,

[0058] FK represents the preliminary sorting result of the features in the feature set.

[0059] FK i It represents the integrated feature ranking result of n feature selection algorithms corresponding to k different features.

[0060] f n,k It represents the kth feature of the nth feature selection algorithm integrated feature.

[0061] ω αi represents the performance weight set of each feature selection algorithm,

[0062] ω α1 represents the performance weight of the first feature selection algorithm,

[0063] ω α2 represents the performance weight of the second feature selection algorithm,

[0064] ω αn Represents the performance weight of the nth feature selection algorithm;

[0065] S7) scoring the importance of the ignored important features, calculating the weights of the ignored important features, weighting the ignored important features, adding the ignored important features after the weighted scores to the preliminary feature sorting, and re-sorting, setting a filtering threshold for screening, filtering the features below the filtering threshold, and adding the features above the filtering threshold to the feature set, to obtain the optimal feature subset as the final result of this integrated feature selection.

[0066] Further, in S54), the stability weight of each feature selection algorithm is obtained by the following formula:

[0067]

[0068] In the formula,

[0069] ω i1 represents the stability weight of each feature selection algorithm,

[0070] Sr i ′ represents Sr i After normalization,

[0071] Sr i represents the stability mean of the same feature selection algorithm,

[0072] i represents the number of feature selection algorithms;

[0073] The accuracy weight of each feature selection algorithm is obtained by the following formula

[0074]

[0075] In the formula,

[0076] ω i2 represents the accuracy weight of each feature selection algorithm,

[0077] MAE i ′ represents MAE i After normalization,

[0078] MAE i represents the mean prediction accuracy of the same feature selection algorithm,

[0079] i represents the number of feature selection algorithms;

[0080] The computational complexity weight of each feature selection algorithm is obtained by the following formula

[0081]

[0082] In the formula,

[0083] ω i3represents the computational complexity weight of each feature selection algorithm,

[0084] T i ′ represents T i After normalization,

[0085] T i represents the mean time consumed by the same feature selection algorithm,

[0086] i represents the number of feature selection algorithms;

[0087] The performance weight of each feature selection algorithm is obtained by the following formula

[0088]

[0089] In the formula,

[0090] ω ai represents the performance weight of each feature selection algorithm,

[0091] ω i1 represents the stability weight of the same feature selection algorithm,

[0092] ω i2 represents the accuracy weight of the same feature selection algorithm,

[0093] ω i3 represents the computational complexity weight of the same feature selection algorithm,

[0094] i represents the number of feature selection algorithms.

[0095] Furthermore, in S54), Sr i Data normalization, MAE i Data normalization, and T i Data normalization uses the Z-score normalization method. The specific formula is as follows

[0096]

[0097] In the formula,

[0098] x is the initial value of all sample data,

[0099] u is the mean of all sample data,

[0100] σ is the standard deviation of all sample data,

[0101] is the normalized value of all sample data.

[0102] The advantages of the present invention are:

[0103] 1. The present invention reduces the influence of different training data on the results of feature selection algorithms based on the distribution characteristics of data sets in different periods and working conditions, while ensuring data diversity;

[0104] 2. The present invention reduces the influence of the performance weight of the feature selection algorithm on the results of the aggregation process from the three performance evaluation criteria of stability evaluation criteria, prediction performance evaluation criteria, and computational complexity evaluation criteria, and increases the stability of feature selection and the probability of obtaining the optimal feature subset;

[0105] 3. The present invention performs weighted processing on ignored important features, assigning higher weights to features in important areas and lower weights to features in unimportant areas, thereby ensuring that important features are selected;

[0106] The present invention is a method for improving the stability of feature selection for performance prediction and ensemble learning of production processes. On the one hand, the method is more comprehensive in the selection of data sets, and reduces the influence of different training data on the results of feature selection algorithms while ensuring data diversity. On the other hand, compared with other ensemble methods, the proposed method has a higher probability of selecting a stable optimal feature subset, and can screen out a stable feature subset with high prediction performance, thus solving the problem of unstable feature selection. BRIEF DESCRIPTION OF THE DRAWINGS

[0107] Figure 1 The present invention is a flow chart of the feature selection stability improvement method for performance prediction and integrated learning of production processes. DETAILED DESCRIPTION

[0108] The present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments.

[0109] In the description of the present invention, it is necessary to understand that the terms "length", "width", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside", etc., indicating the orientation or position relationship, are based on the orientation or position relationship shown in the drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation on the invention.

[0110] like Figure 1 As shown, the method for improving the stability of feature selection for performance prediction and integrated learning of a production process of the present invention comprises the following steps:

[0111] S1) Collect real-time data of the production process and pre-process the real-time data.

[0112] Specifically, a data acquisition interface is used to communicate with a production control system, a resource planning system, etc. to obtain real-time data of the production process.

[0113] Specifically, real-time data is collected by reading sensors at each production line location to obtain relevant production data.

[0114] Specifically, the preprocessing includes filling, deleting, and data normalization. The data normalization refers to removing the unit restriction of the data and converting the data into dimensionless pure values ​​to facilitate the comparison and weighting of indicators of different units or magnitudes. The data normalization adopts the Z-score normalization processing method. The specific formula is as follows:

[0115]

[0116] In the formula,

[0117] x is the initial value of all sample data,

[0118] u is the mean of all sample data,

[0119] σ is the standard deviation of all sample data,

[0120] is the normalized value of all sample data.

[0121] S2) Use statistical methods to divide the preprocessed data according to different production conditions to obtain data sets under different production conditions, and use orthogonal experiment method to classify the divided data sets to ensure the average distribution of each influencing factor and each level of the orthogonal experiment, achieve uniform dispersion of influencing factors and levels, and obtain classified data sets.

[0122] Specifically, the influencing factors in the orthogonal experiment include time period, working conditions and varieties, and each influencing factor has 4 levels.

[0123] First, we need to determine the influencing factors in actual production (in this embodiment, the influencing factors A, B, and C are production period, working condition, and variety, respectively) and the number of levels of each influencing factor. According to the number of levels and the number of influencing factors, the type of orthogonal table to be used is determined. The code of the orthogonal table is Lm(s y ), where L represents the orthogonal table, m represents the number of experiments, s represents the number of levels of the influencing factors, and y represents the number of influencing factors. Assuming that the number of levels of each influencing factor in this embodiment is 4, 16 batches of data sets will be combined for the experiment, and the orthogonal experimental design table is shown in Table 1.

[0124] Table 1 Orthogonal test design table

[0125] serial number A B C 1 A1 B1 C1 2 A1 B2 C2 3 A1 B3 C3 4 A1 B4 C4 5 A2 B1 C2 6 A2 B2 C1 7 A2 B3 C4 8 A2 B4 C3 9 A3 B1 C3 10 A3 B2 C4 11 A3 B3 C1 12 A3 B4 C2 13 A4 B1 C4 14 A4 B2 C3 15 A4 B3 C2 16 A4 B4 C1

[0126] The data set classification is divided according to the influencing factors. In this embodiment, the main influencing factors are time period, working condition and variety. Time period: Since the production environment changes with the seasons, the environmental variables are quite different in different seasons, so the production data is divided into time periods according to different seasons. Working condition: The initial parameters of different batches of raw materials are different, and the required processing volume has also changed, so the production data is divided into working conditions according to different batches of raw materials. Variety: The production environment required for different varieties of raw materials varies, so in order to reduce the impact of this aspect, the production data is divided into varieties according to different varieties of raw materials. As shown in Table 2.

[0127] Table 2 Dataset influencing factors

[0128]

[0129] S3) Perform multicollinearity diagnosis on the features in each classification data set, set a correlation coefficient threshold, if the feature correlation coefficient is greater than the correlation coefficient threshold, then the feature multicollinearity is determined to be strong, if the feature correlation coefficient is less than the correlation coefficient threshold, then the feature multicollinearity is determined to be weak; according to the multicollinearity diagnosis result, the features with weak multicollinearity are divided into independent feature data sets, and the features with strong multicollinearity are divided into several non-independent feature data sets; for each non-independent feature data set, only one feature with the highest correlation with the target feature is retained as an independent feature; several independent features are obtained through several non-independent feature data sets, and the several independent features are added to the independent feature data set to obtain feature independent data sets.

[0130] After the data sets are classified, these classified data sets will be processed for feature multicollinearity test. j Take the multicollinearity processing of the features in (j=1, 2, ..., m) as an example, use correlation analysis to perform multicollinearity test on one of the features, and set the correlation coefficient threshold K N , the correlation coefficient is greater than K N , the feature is judged to have strong multicollinearity, otherwise it is judged to have weak multicollinearity. According to the multicollinearity judgment result, the features with weak multicollinearity are divided into independent feature data sets F a (1, 2, ..., n), divide the features with strong multicollinearity into several non-independent feature data sets F b1 (1, 2...i), F b2 (1, 2...j),...F bn (1, 2...f). Each non-independent feature data set only retains one feature with the highest correlation with the target feature, and obtains several independent features b 1i , b2j , ...b nf , several independent features b 1i , b 2j , ...b nf Add independent feature dataset F a (1, 2, ..., n), obtain the feature independent data set F a (1, 2, ..., n, b 1i , b 2j , ...b nf ).

[0131] S4) Repeat step S3) to process all classification data sets into feature-independent data sets.

[0132] S5) For each feature independent data set, different feature selection algorithms are used to select features, obtain feature subsets, and evaluate the performance weights of different feature selection algorithms through all feature subsets. Different feature selection algorithms are sorted according to the performance weights, wherein the feature selection algorithm with a higher performance weight has a higher priority in the integration process; the feature selection algorithm with a lower performance weight has a lower priority in the integration process.

[0133] The specific evaluation steps of the performance weights of different feature selection algorithms are as follows:

[0134] S51) Calculate the stability of each feature selection algorithm on each feature independent data set, and take the stability mean as the stability evaluation criterion of the feature selection algorithm.

[0135] After being processed into feature-independent datasets, these feature-independent datasets will be used to evaluate the performance of the feature selector. Taking the performance weight evaluation of one of the feature selection algorithms Ai (i = 1, 2, ..., n) as an example, the m feature-independent datasets obtained by screening and combining are used to evaluate the performance of each feature-independent dataset F. j (j=1, 2, ..., m) using A i Perform feature selection to obtain feature subset FS j (j=1, 2, ..., m) m feature subsets FS j Compare each pair to obtain m×(m-1) / 2 similarity results.

[0136] Specifically, the stability of each feature selection algorithm on different feature independent data sets is obtained by the following formula:

[0137]

[0138] In the formula,

[0139] S R(r, r′) represents the stability coefficient of each feature selection algorithm on the feature independent data sets D and D′, which is used to measure the stability of feature selection. R (r, r′) = [-1, 1],

[0140] r represents the result of running the same feature selection algorithm on the feature-independent data set D,

[0141] r′ represents the result of running the same feature selection algorithm on the feature-independent data set D′,

[0142] r i represents the ranking of the running results of the i-th feature selection algorithm on the feature-independent data set D,

[0143] r′ i represents the ranking of the results of the i-th feature selection algorithm on the feature-independent data set D′,

[0144] m represents the number of feature subsets,

[0145] i represents the number of feature selection algorithms.

[0146] The final stability is taken as the average value. Specifically, the stability average is obtained by the following formula:

[0147]

[0148] In the formula,

[0149] Sr represents the stability mean of each feature selection algorithm,

[0150] S R (r, r′) represents the stability coefficient of the same feature selection algorithm on feature-independent data sets D and D′, which is used to measure the stability of feature selection.

[0151] r represents the result of running the same feature selection algorithm on the feature-independent data set D,

[0152] m represents the number of feature subsets,

[0153] m×(m-1) / 2 represents the number of similar results obtained by comparing m feature subsets pairwise.

[0154] Since the result of the feature selection algorithm of the present invention is mainly a ranking result, the Spearman rank correlation coefficient is selected to measure the stability. If Sr=1, the ranking results are exactly the same, and if Sr=-1, the ranking results are completely different.

[0155] S52) Construct a prediction model based on data-driven, input the feature subsets screened by each feature selection algorithm on different feature independent data sets into the prediction model for prediction, and take the mean prediction accuracy as the prediction performance evaluation criterion of the feature selection algorithm.

[0156] Specifically, the feature subset FS j As input, m data-driven models are constructed and the target values ​​(such as product quality, remaining life of parts, etc.) are predicted. m error values ​​are calculated as prediction errors and their average value is taken to evaluate the prediction accuracy of the feature selection algorithm.

[0157] Specifically, the mean prediction accuracy of each feature selection algorithm is obtained by

[0158]

[0159] In the formula,

[0160] MAE represents the mean prediction accuracy of each feature selection algorithm.

[0161] N l Indicates the number of values,

[0162] L represents the actual value,

[0163] L′ represents the predicted value,

[0164] |LL′| represents the prediction error value.

[0165] S53) Calculate the time consumed by each feature selection algorithm in the feature selection process, and take the average of the consumed time as the computational complexity evaluation criterion of the feature selection algorithm.

[0166] S54) Calculate the stability weight according to the mean stability of each feature selection algorithm, calculate the accuracy weight according to the mean prediction accuracy of each feature selection algorithm, calculate the computational complexity weight according to the mean time consumed by each feature selection algorithm, and calculate the performance weight of each feature selection algorithm by combining the stability weight, accuracy weight, and computational complexity weight.

[0167] Specifically, the stability weight of each feature selection algorithm is obtained by the following formula

[0168]

[0169] In the formula,

[0170] ω i1 represents the stability weight of each feature selection algorithm,

[0171] Sr i ′ means Sri has been normalized.

[0172] Sr i represents the stability mean of the same feature selection algorithm,

[0173] i represents the number of feature selection algorithms;

[0174] The accuracy weight of each feature selection algorithm is obtained by the following formula

[0175]

[0176] In the formula,

[0177] ω i2 represents the accuracy weight of each feature selection algorithm,

[0178] MAE i ′ represents MAE i After normalization,

[0179] MAE i represents the mean prediction accuracy of the same feature selection algorithm,

[0180] i represents the number of feature selection algorithms;

[0181] The computational complexity weight of each feature selection algorithm is obtained by the following formula

[0182]

[0183] In the formula,

[0184] ω i3 represents the computational complexity weight of each feature selection algorithm,

[0185] T i ′ represents T i After normalization,

[0186] T i represents the mean time consumed by the same feature selection algorithm,

[0187] i represents the number of feature selection algorithms;

[0188] The performance weight of each feature selection algorithm is obtained by the following formula

[0189]

[0190] In the formula,

[0191] ω ai represents the performance weight of each feature selection algorithm,

[0192] ω i1 represents the stability weight of the same feature selection algorithm,

[0193] ω i2 represents the accuracy weight of the same feature selection algorithm,

[0194] ω i3 represents the computational complexity weight of the same feature selection algorithm,

[0195] i represents the number of feature selection algorithms.

[0196] Specifically, Sr i Data normalization, MAE i Data normalization, and T i Data normalization uses the Z-score normalization method. The specific formula is as follows

[0197]

[0198] In the formula,

[0199] x is the initial value of all sample data,

[0200] u is the mean of all sample data,

[0201] σ is the standard deviation of all sample data,

[0202] is the normalized value of all sample data.

[0203] Sr i ′ and T i The larger the proportion of ′ value, the higher the weight, MAE i The larger the proportion of ′, the lower the weight, so the exponential function e is used. x Perform numerical conversion and finally select the stability Sr of the algorithm according to each feature i The proportion of ′ is used to obtain the stability weight ω i1 , prediction accuracy MAE i ′Get the accuracy weight ω i2 , and the time consumed T i ′Get the computational complexity weight ω i3 , combined to calculate the performance weight ω of each feature selection algorithm ai .

[0204] S6) Sort and score the positions of the k A features selected from all feature subsets in all feature selection algorithms, then add the k ranking scores corresponding to the k A features by the same feature selection algorithm to obtain the final score of the A feature, and use the final score of the A feature as the integrated feature ranking result of the feature selection algorithm corresponding to the A feature, finally multiply the integrated feature ranking result of each feature selection algorithm corresponding to the A feature by the performance weight of the corresponding feature selection algorithm and add them together to obtain the weighted score of the A feature; re-sort the feature set according to the weighted score of each feature to obtain a preliminary feature ranking.

[0205] Specifically, according to Borda count, each feature is ranked in the sorting result A. i (F j ) are voted and scored in the positions. Assume that the first-ranked feature gets M points, the second-ranked feature gets M-1 points, and so on. The last-ranked feature gets 1 point.

[0206] Then the feature ranking results selected by the same feature selection algorithm on different feature subsets are integrated, that is, the scores of each feature in different ranking results are added to obtain the final score of the feature, and the integrated feature ranking result FK corresponding to the feature of the single feature selection algorithm is obtained. i =(f1, f2, ..., f k ), according to the scores of k features in different ranking results multiplied by the weight coefficient ω αi =|ω α1 ω α2 … ω αn | T Then add them together to get the weighted score of each feature.

[0207] Finally, the feature set is reordered according to the weighted score of each feature to obtain the preliminary feature ranking result FK. Specifically, the preliminary feature ranking result is obtained by the following formula:

[0208]

[0209] In the formula,

[0210] FK represents the preliminary sorting result of the features in the feature set.

[0211] FK i It represents the integrated feature ranking result of n feature selection algorithms corresponding to k different features.

[0212] f n，k It represents the kth feature of the nth feature selection algorithm integrated feature.

[0213] ω αirepresents the performance weight set of each feature selection algorithm,

[0214] ω α1 represents the performance weight of the first feature selection algorithm,

[0215] ω α2 represents the performance weight of the second feature selection algorithm,

[0216] ω αn Represents the performance weight of the nth feature selection algorithm.

[0217] S7) scoring the importance of the ignored important features, calculating the weights of the ignored important features, weighting the ignored important features, adding the ignored important features after the weighted scores to the preliminary feature sorting, and re-sorting, setting a filtering threshold for screening, filtering the features below the filtering threshold, and adding the features above the filtering threshold to the feature set, to obtain the optimal feature subset as the final result of this integrated feature selection.

[0218] Specifically, the empirical feature area is divided according to the data characteristics and correlation in the production process. Although the preliminary feature sorting result FK is obtained through the above step S6), some key features may be ignored due to the missing, fuzzy, and noisy problems in the actual production data, so it is necessary to weight the empirical features to ensure that the important features ignored by the algorithm can be selected into the optimal feature subset.

[0219] Score the importance and use the hierarchical analysis method to calculate the weight of each feature of the lowest level indicator ω bl , weight the empirical feature scores obtained in step S6). After adjusting the empirical feature scores, reorder FK and set the filter threshold C for screening, filter the features below the filter threshold C, and add the features above the filter threshold C to the feature set. The feature set finally selected will be the final result of this integrated feature selection.

[0220] As shown in Table 3, this embodiment combines the integration strategy of Borda count to aggregate multiple feature selection algorithm results.

[0221] Table 3 Weights of empirical characteristics

[0222]

[0223] The above embodiments are preferred implementation modes of the present invention, but the implementation modes of the present invention are not limited to the above embodiments. Any other changes, modifications, substitutions, combinations, and simplifications that do not deviate from the spirit and principles of the present invention should be equivalent replacement methods and are included in the protection scope of the present invention.

Claims

1. A method for improving the stability of feature selection for performance prediction and ensemble learning in production processes, characterized in that: The steps include: S1) Collecting real-time data of the production process and preprocessing the real-time data; Real-time data collection is done by reading sensors at each production line location to obtain relevant production data; The preprocessing includes filling, deleting, and data normalization. The data normalization refers to removing the unit restrictions of the data and converting the data into dimensionless pure values ​​to facilitate comparison and weighting of indicators of different units or magnitudes. The data normalization adopts the Z-score normalization method. The specific formula is as follows: In the formula, x is the initial value of all sample data, u is the mean of all sample data, σ is the standard deviation of all sample data, is the normalized value of all sample data; S2) using statistical methods to divide the preprocessed data according to different production conditions to obtain data sets under different production conditions, and using orthogonal test methods to classify the divided data sets to ensure the average distribution of each influencing factor and each level of the orthogonal test, achieve uniform dispersion of the influencing factors and levels, and obtain classified data sets; In S2), the influencing factors in the orthogonal experiment include time period, working conditions and varieties, and each influencing factor has 4 levels; S3) performing multicollinearity diagnosis on the features in each classification data set, setting a correlation coefficient threshold, if the feature correlation coefficient is greater than the correlation coefficient threshold, it is determined that the feature multicollinearity is strong, if the feature correlation coefficient is less than the correlation coefficient threshold, it is determined that the feature multicollinearity is weak; according to the multicollinearity diagnosis result, the features with weak multicollinearity are divided into independent feature data sets, and the features with strong multicollinearity are divided into several non-independent feature data sets; for each non-independent feature data set, only one feature with the highest correlation with the target feature is retained as an independent feature; Obtain a number of independent features through a number of non-independent feature data sets respectively, and add the several independent features into the independent feature data set to obtain a feature independent data set; S4) repeating step S3) to process all classification data sets into feature-independent data sets; S5) for each feature independent data set, using different feature selection algorithms to perform feature selection, obtain feature subsets, and evaluate the performance weights of different feature selection algorithms through all feature subsets; The specific evaluation steps of the performance weights of different feature selection algorithms are as follows: S51) calculating the stability of each feature selection algorithm on each feature independent data set, and taking the stability mean as the stability evaluation criterion of the feature selection algorithm; The stability of each feature selection algorithm on different feature independent data sets is obtained by the following formula In the formula, S R (r, r′) represents the stability coefficient of each feature selection algorithm on the feature independent data sets D and D′, which is used to measure the stability of feature selection. R (r, r′) = [-1, 1], r represents the result of running the same feature selection algorithm on the feature-independent data set D, r′ represents the result of running the same feature selection algorithm on the feature-independent data set D′, r i represents the ranking of the running results of the i-th feature selection algorithm on the feature-independent data set D, r′ i represents the ranking of the results of the i-th feature selection algorithm on the feature-independent dataset D', m represents the number of feature subsets, i represents the number of feature selection algorithms; The stability mean is obtained by the following formula In the formula, Sr represents the stability mean of each feature selection algorithm, S r (r, r′) represents the stability coefficient of the same feature selection algorithm on feature-independent data sets D and D′, which is used to measure the stability of feature selection. r represents the result of running the same feature selection algorithm on the feature-independent data set D, m represents the number of feature subsets, m×(m-1) / 2 means the number of similar results obtained by comparing m feature subsets pairwise; S52) constructing a prediction model based on data-driven, inputting the feature subsets selected by each feature selection algorithm on different feature independent data sets into the prediction model for prediction, and taking the mean of the prediction accuracy as the prediction performance evaluation criterion of the feature selection algorithm; The mean prediction accuracy of each feature selection algorithm is obtained by In the formula, MAE represents the mean prediction accuracy of each feature selection algorithm. N l Indicates the number of values, L represents the actual value, L′ represents the predicted value, |LL′| represents the prediction error value; S53) calculating the time consumed by each feature selection algorithm in the feature selection process, and taking the average of the consumed time as the computational complexity evaluation criterion of the feature selection algorithm; S54) calculating a stability weight according to the mean stability of each feature selection algorithm, calculating an accuracy weight according to the mean prediction accuracy of each feature selection algorithm, calculating a computational complexity weight according to the mean consumption time of each feature selection algorithm, and calculating a performance weight of each feature selection algorithm by combining the stability weight, the accuracy weight, and the computational complexity weight; S6) sorting and scoring the positions of the k A features selected from all feature subsets in all feature selection algorithms, and then adding the k sorting scores corresponding to the k A features by the same feature selection algorithm to obtain the final score of the A feature, and using the final score of the A feature as the integrated feature sorting result corresponding to the A feature by the feature selection algorithm, and finally multiplying the integrated feature sorting result corresponding to the A feature by each feature selection algorithm by the performance weight of the corresponding feature selection algorithm and adding them to obtain the weighted score of the A feature; re-sorting the feature set according to the weighted score of each feature to obtain a preliminary feature sorting; The preliminary ranking result of the features is obtained by the following formula In the formula, FK represents the preliminary sorting result of the features in the feature set. FK i It represents the integrated feature ranking result of n feature selection algorithms corresponding to k different features. f n,k It represents the kth feature of the nth feature selection algorithm integrated feature. ω αi represents the performance weight set of each feature selection algorithm, ω α1 represents the performance weight of the first feature selection algorithm, ω α2 represents the performance weight of the second feature selection algorithm, ω αn Represents the performance weight of the nth feature selection algorithm; S7) scoring the importance of the ignored important features, calculating the weights of the ignored important features, weighting the ignored important features, adding the ignored important features after the weighted scores to the preliminary feature sorting, and re-sorting, setting a filtering threshold for screening, filtering the features below the filtering threshold, and adding the features above the filtering threshold to the feature set, to obtain the optimal feature subset as the final result of this integrated feature selection.

2. The method for improving the stability of feature selection for performance prediction and ensemble learning for production processes according to claim 1, characterized in that: S54), the stability weight of each feature selection algorithm is obtained by the following formula In the formula, ω i1 represents the stability weight of each feature selection algorithm, Sr i ′ represents Sr i After normalization, Sr i represents the stability mean of the same feature selection algorithm, i represents the number of feature selection algorithms; The accuracy weight of each feature selection algorithm is obtained by the following formula In the formula, ω i2 represents the accuracy weight of each feature selection algorithm, MAE i ′ represents MAE i After normalization, MAE i represents the mean prediction accuracy of the same feature selection algorithm, i represents the number of feature selection algorithms; The computational complexity weight of each feature selection algorithm is obtained by the following formula In the formula, ω i3 represents the computational complexity weight of each feature selection algorithm, T i ′ represents T i After normalization, T i represents the mean time consumed by the same feature selection algorithm, i represents the number of feature selection algorithms; The performance weight of each feature selection algorithm is obtained by the following formula In the formula, ω ai represents the performance weight of each feature selection algorithm, ω i1 represents the stability weight of the same feature selection algorithm, ω i2 represents the accuracy weight of the same feature selection algorithm, ω i3 represents the computational complexity weight of the same feature selection algorithm, i represents the number of feature selection algorithms.

3. The method for improving the stability of feature selection for performance prediction and ensemble learning for production processes according to claim 2, characterized in that: S54), Sr i Data normalization, MAE i Data normalization, and T i Data normalization uses the Z-score normalization method. The specific formula is as follows In the formula, x is the initial value of all sample data, u is the mean of all sample data, σ is the standard deviation of all sample data, is the normalized value of all sample data.

Images