Self-adaptive mixed feature selection method for industrial big data

Abstract

According to the industrial big data-oriented adaptive mixed feature selection method provided by the invention, efficient and accurate feature selection is realized through dynamic evaluation and an adaptive adjustment mechanism. According to the method, multiple feature selection methods are introduced, a dynamic evaluation mechanism is combined, the advantages of different methods are comprehensively utilized, the features are screened and optimized from multiple angles, and the quality of a final feature set and the correlation with target variables are improved; through dynamic evaluation of performance contribution and volatility of each feature selection method, the participation weight of each method is adaptively adjusted, so that the feature selection process can be flexibly adjusted according to data characteristics, and stable performance can still be maintained in a complex and variable data scene. According to the method, the problem that a traditional feature selection method is insufficient in precision, efficiency and robustness in a high-dimensional data scene is solved, and an efficient and reliable solution is provided for feature optimization and application of industrial big data.

Description

Technical Field

[0001] The present invention relates to the technical field of industrial data processing, and particularly relates to an adaptive hybrid feature selection method for industrial big data, which is particularly applicable to feature screening and optimization problems in industrial big data analysis. Background Art

[0002] With the rapid development of industrial big data technology, high-dimensional data analysis has become a research hotspot. However, high-dimensional data usually has the following characteristics: high dimensionality, few samples, much noise, and rich redundant information. These characteristics bring great challenges to data analysis. Especially in modeling and prediction tasks, feature selection, as a dimensionality reduction technology, is widely adopted. Its purpose is to screen out features highly correlated with the target variable from high-dimensional features, reduce noise interference, and improve the performance and stability of the model. Traditional feature selection methods are mainly divided into three categories: Filter methods are independent of learning algorithms and evaluate the importance of features by calculating the correlation or dependence between features and the target variable. This method is simple and efficient, but it is easy to ignore the interaction information between features; Embedded methods combine learning algorithms for feature selection, can consider the interaction between features, but usually rely on specific models and have poor generality; Wrapper methods take the model performance as the optimization goal and search for the best feature subset iteratively. Although the selection effect is good, the computational cost is high and it is difficult to process high-dimensional data. The above methods may be effective in a single application scenario, but for the high-dimensional small-sample data widely existing in industrial big data, their performance usually cannot meet the actual needs. Therefore, hybrid feature selection methods have become a research hotspot, and by combining the advantages of multiple feature selection methods, the accuracy and robustness of feature selection are improved. However, the existing hybrid methods usually lack the ability of dynamic adjustment and cannot be flexibly optimized according to specific data characteristics and the actual contributions of feature selection methods, resulting in unstable and inefficient selection results. Therefore, there is an urgent need for a new solution to solve the above technical problems. Summary of the Invention

[0003] To solve the above technical problems, the present invention provides an adaptive hybrid feature selection method for industrial big data, aiming to provide an efficient and reliable solution for feature selection and optimization of high-dimensional data through efficient and flexible feature selection.

[0004] To solve the above technical problems, the present invention provides the following technical solution: An adaptive hybrid feature selection method for industrial big data, the method includes the following steps,

[0005] First, a hybrid feature selection method is proposed. Multiple predefined feature selection methods are used to evaluate the feature set, and the candidate feature subsets selected by each method are combined to obtain a candidate feature set;

[0006] Secondly, a dynamic evaluation method for feature selection is proposed to dynamically evaluate the performance contribution and performance volatility of each feature selection method during the iteration process;

[0007] Then, an adaptive adjustment mechanism is proposed to adaptively adjust the participation degree of each method in the next round of iteration during the iteration process;

[0008] Finally, repeat the above steps, add the candidate feature set obtained in each round to the final feature set, and stop the iteration when the final feature set reaches the predetermined size to achieve the final feature selection.

[0009] As a preferred solution of the present invention, wherein: a hybrid feature selection method is proposed, which uses multiple predefined feature selection methods to evaluate the feature set, combines the candidate feature subsets selected by each method, and obtains a candidate feature set, including the following steps:

[0010] First, obtain the complete original feature set F = {f 1 , f 2 , … f n}.

[0011] Then, run the hybrid feature selection method in parallel to independently evaluate the initial feature set F, and each method generates a candidate feature subset respectively.

[0012] Among them, the hybrid feature selection method includes a feature selection method M 1 based on the Pearson correlation coefficient, and the calculation formula is as follows:

[0013]

[0014] Among them, x i , y i are the values of the i-th sample on the feature X and the target variable Y respectively, are the means of all samples on the feature X and the target variable Y respectively, n is the total number of samples, and a candidate feature subset with the number of generated features being is generated

[0015] Among them, the hybrid feature selection method includes a feature selection method M 2 based on information gain, and the calculation formula is as follows:

[0016] IG(X, Y) = H(Y) - H(Y|X)

[0017] Among them, H(Y) represents the uncertainty of the target variable Y, and the calculation formula is as follows:

[0018]

[0019] Among them, P(y) represents the probability that the target variable takes the value y.

[0020] Among them, H(Y|X) represents the uncertainty of the target variable Y when the feature X is known, and the calculation formula is as follows:

[0021]

[0022] Among them, P(x) represents the probability that the feature X takes the value x, P(y|x) represents the conditional probability that Y is y under the condition that X is x, and the number of generated features is Candidate feature subset

[0023] Among them, the hybrid feature selection method includes the feature selection method M based on Lasso regularization 3 , and the calculation formula of its objective function is as follows:

[0024]

[0025] Among them, N represents the total number of samples, y i represents the actual value of the i-th sample, represents the predicted value of the i-th sample, λ represents the regularization hyperparameter that controls the strength of the penalty term, p represents the number of features, is the L1 norm, which represents the sum of the absolute values of the feature coefficients, and the number of generated features is Candidate feature subset

[0026] Finally, collect the candidate feature subsets generated by each feature selection method to obtain the candidate feature set C t of this round of iteration, and combine C t with the final feature set S t-1 in the previous round of iteration to form the final feature set S t .

[0027] As a preferred embodiment of the present invention, among them: the feature selection dynamic evaluation method dynamically evaluates the performance contribution and performance volatility of each feature selection method during the iteration process, including the following steps:

[0028] First, according to the currently obtained c j ∈C t , combine it with the final feature set S t-1 in the previous round of iteration to form the feature set S′ t =S t ∪{c j}.

[0029] Then, use the feature subset S′ tTrain a support vector machine model, and the calculation formula is as follows:

[0030] f(x) = sign(W T x + b)

[0031] where x is the sample input vector, which contains all the features in the feature subset S′ t ; w is the weight vector of the support vector machine, representing the linear combination of features; b is the bias term; f(x) is the predicted output of the model, representing the predicted value of the sample x belonging to a certain category. The training objective of the support vector machine is to find a decision boundary that maximizes the margin, such that:

[0032]

[0033] where y i is the true label of x i , and w and b are the parameters to be optimized. Calculate the classification error rate Err(S′ t ), and the calculation formula is as follows:

[0034]

[0035] where n is the number of samples, is the predicted label, y i is the true label, is the indicator function, and when the prediction is incorrect, the function value is 1, otherwise it is 0. Calculate the error change between the current feature set S t and the new feature set S′ j that contains the candidate feature c t to evaluate the contribution of the feature selection method, and the calculation formula is as follows:

[0036]

[0037] If the error decreases (i.e., ), it indicates that the method improves the performance of the model; otherwise, it means that the method has little improvement on the model.

[0038] Next, calculate the performance contribution index i of the feature selection method M , and the calculation formula is as follows:

[0039]

[0040] Finally, calculate the performance volatility index i of the feature selection method M , and the calculation formula is as follows:

[0041]

[0042] where is method M i The performance contribution index in the most recent k iterations is method M i The average value of the performance contribution index in the most recent k iterations

[0043] As a preferred solution of the present invention, wherein: the adaptive adjustment mechanism adaptively adjusts the participation degree of each method in the next iteration during the iteration process, including the following steps:

[0044] First, select method M according to the obtained characteristics i The performance contribution index of and the performance volatility index Introduce a two-factor weight smoothing mechanism. Through historical weight memory and two-parameter adjustment, avoid sudden weight changes, and calculate the participation degree weight of this method The calculation formula is as follows:

[0045]

[0046] where β is the contribution factor smoothing coefficient, γ is the fluctuation suppression intensity coefficient, represents the weight value of the previous iteration

[0047] Then, according to the participation degree weight of the feature selection method M i introduce a dynamic candidate feature number allocation mechanism to adjust the number of features in the candidate feature set generated by this method in the next iteration The calculation formula is as follows: The number of features of The calculation formula is as follows:

[0048]

[0049] where N t is the total number of candidate features in this iteration, ceil is the ceiling operation, when perform adjustments according to the following priorities: first ensure the method with the largest value; secondarily ensure the method with the lowest average value in the current round

[0050] Finally, adaptively adjust the proportion of candidate features generated by each method in the candidate feature set C in the next iteration t to flexibly adjust the participation degree of each method

[0051] An electronic device, including a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that: when the processor executes the program, it implements the adaptive hybrid feature selection method for industrial big data described above​

[0052] A computer-readable storage medium stores computer instructions thereon, characterized in that: when the computer instructions are executed by a processor, the adaptive hybrid feature selection method for industrial big data described above is implemented.

[0053] Advantages of the present invention: In view of the characteristics of high-dimensionality, small samples, and multi-noise of industrial big data, the present invention proposes a dynamic adaptive hybrid feature selection method. By integrating the advantages of multiple feature selection strategies and combining real-time performance evaluation and weight adjustment mechanisms, the problems of poor flexibility, low computational efficiency, and strong parameter dependence of traditional methods are solved. Specifically, during the feature screening process, the system can dynamically optimize the participation weights of different methods according to the data distribution characteristics and model feedback, avoiding insufficient scenario adaptability caused by fixed fusion strategies; through parallel processing and pre-screening mechanisms, while ensuring the accuracy of multi-dimensional feature evaluation, the consumption of computing resources is significantly reduced; in addition, based on the historical performance volatility of the feature selection method, the decision weights of unstable methods are adaptively suppressed, reducing the need for manual parameter intervention and improving the stability of cross-scenario migration. The present invention deeply combines the dynamic evaluation mechanism with the machine learning model, synchronously optimizes the model prediction ability during the feature selection process, effectively eliminates the interference of redundant features, and thus realizes efficient and robust feature optimization in complex industrial data scenarios, providing reliable technical support for applications such as equipment fault prediction and process parameter analysis. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following will briefly introduce the drawings required for the description of the embodiments. Obviously, the following drawings are only some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings. Among them:

[0055] Figure 1 It is a schematic diagram of the basic process of an adaptive hybrid feature selection method for industrial big data provided by an embodiment of the present invention;

[0056] Figure 2 It is a schematic diagram of the specific implementation manner of an adaptive hybrid feature selection method for industrial big data provided by an embodiment of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0057] Embodiment 1

[0058] Referring to Figure 1 - Figure 2 , which is an embodiment of the present invention, an adaptive hybrid feature selection method for industrial big data is provided, including the following steps:

[0059] S1: Propose a hybrid feature selection method. Use multiple predefined feature selection methods to evaluate the feature set, combine the candidate feature subsets selected by each method, and obtain a candidate feature set;

[0060] Furthermore, obtain the complete original feature set F = {f 1 , f 2 , … f n}.

[0061] Furthermore, run the hybrid feature selection method in parallel to independently evaluate the initial feature set F, and each method generates a candidate feature subset respectively.

[0062] Among them, the hybrid feature selection method includes a feature selection method M based on the Pearson correlation coefficient 1 , and the calculation formula is as follows:

[0063]

[0064] where x i , y i are the values of the i-th sample on the feature X and the target variable Y respectively, are the means of all samples on the feature X and the target variable Y respectively, n is the total number of samples, and a candidate feature subset with the number of generated features being is generated

[0065] Among them, the hybrid feature selection method includes a feature selection method M based on information gain 2 , and the calculation formula is as follows:

[0066] IG(X,Y) = H(Y) - H(Y|X)

[0067] where H(Y) represents the uncertainty of the target variable Y, and the calculation formula is as follows:

[0068]

[0069] where P(y) represents the probability that the target variable takes the value of y.

[0070] where H(Y|X) represents the uncertainty of the target variable Y given the feature X, and the calculation formula is as follows:

[0071]

[0072] where P(x) represents the probability that the feature X takes the value of x, P(y|x) represents the conditional probability that Y is y given X is x, and a candidate feature subset with the number of generated features being is generated

[0073] Among them, the hybrid feature selection method includes the feature selection method M based on Lasso regularization 3 , and the calculation formula of its objective function is as follows:

[0074]

[0075] Among them, N represents the total number of samples, y i represents the actual value of the i-th sample, represents the predicted value of the i-th sample, λ represents the regularization hyperparameter that controls the strength of the penalty term, p represents the number of features, is the L1 norm, representing the sum of the absolute values of the feature coefficients, and generating a candidate feature subset with the number of features being

[0076] Furthermore, collect the candidate feature subsets generated by each feature selection method to obtain the candidate feature set C t for this round of iteration, and combine C t with the final feature set S t-1 in the previous round of iteration to form the final feature set S t for the current round.

[0077] S2: Propose a dynamic evaluation method for feature selection to dynamically evaluate the performance contribution and performance volatility of each feature selection method during the iteration process;

[0078] Furthermore, according to the currently obtained c j ∈C t , combine it with the final feature set S t-1 in the previous round of iteration to form the feature set S′ t = S t ∪{c j}.

[0079] Furthermore, use the feature subset S′ t to train a support vector machine model, and the calculation formula is as follows:

[0080] f(x) = sign(W T x + b)

[0081] Among them, x is the sample input vector, which contains all the features in the feature subset S′ t , w is the weight vector of the support vector machine, representing the linear combination of features, b is the bias term, f(x) is the predicted output of the model, representing the predicted value of the sample x belonging to a certain category. The training objective of the support vector machine is to find a decision boundary that maximizes the margin, such that:

[0082] ​

[0083] Among them, y i is the true label of x i and w and b are parameters to be optimized. Calculate the classification error rate Err(S′ t ), and the calculation formula is as follows:

[0084]

[0085] Among them, n is the number of samples, is the predicted label, y i is the true label, is the indicator function. When the prediction is incorrect, the function value is 1, otherwise it is 0. Calculate the current feature set S t and the new feature set S′ j containing the candidate feature c t to evaluate the contribution of the feature selection method. The calculation formula is as follows:

[0086]

[0087] If the error decreases (i.e., ), it means that the method improves the performance of the model. Otherwise, it means that the method has little improvement on the model.

[0088] Furthermore, calculate the performance contribution index i of the feature selection method M The calculation formula is as follows:

[0089]

[0090] Furthermore, calculate the performance volatility index i of the feature selection method M The calculation formula is as follows:

[0091]

[0092] Among them, is the performance contribution index of method M i in the last k iterations, is the average value of the performance contribution index of method M i in the last k iterations.

[0093] S3: Propose an adaptive adjustment mechanism to adaptively adjust the participation degree of each method in the next iteration during the iteration process;

[0094] Furthermore, according to the obtained performance contribution index i of the feature selection method M and the performance volatility index Introduce a two-factor weight smoothing mechanism. Through historical weight memory and two-parameter adjustment, avoid weight mutation and calculate the participation degree weight of this method. The calculation formula is as follows:

[0095]

[0096] Among them, β is the contribution factor smoothing coefficient, and γ is the fluctuation suppression intensity coefficient. represents the weight value of the previous round of iteration.

[0097] Furthermore, according to the participation degree weight of the feature selection method M i introduce a dynamic candidate feature number allocation mechanism to adjust the number of features in the candidate feature set generated by this method in the next round of iteration The calculation formula is as follows: The number of features The calculation formula is as follows:

[0098]

[0099] Among them, N t is the total number of candidate features in this round of iteration, ceil is the ceiling operation. When , adjust according to the following priority: first ensure the method with the largest value; second, ensure the method with the lowest average value in the current round .

[0100] Furthermore, adaptively adjust the proportion of candidate features generated by each method in the candidate feature set C t in the next round of iteration, and flexibly adjust the participation degree of each method.

[0101] S4: Repeat the above steps, add the candidate feature set obtained in each round to the final feature set, and stop the iteration when the final feature set reaches the predetermined size to achieve the final feature selection.

[0102] Example 2

[0103] The present invention provides an adaptive hybrid feature selection method for industrial big data. In order to verify its beneficial effects, scientific demonstration is carried out through specific implementation methods and implementation effects. Experiments are carried out through PYTHON on the WINDOWS11 system. The specific implementation methods are as follows:

[0104] Step 1) Use the APS Failure at Scania Trucks dataset, which contains sensor data and equipment failure information about the equipment. The target variable of the dataset is a binary classification label representing sensor failure, marking whether each record has a failure. The dataset contains 21 sensor data features, covering different measurement items such as temperature, pressure, speed, etc., and contains approximately 13,000 records and 170 features.

[0105] Step 2) Encode the labels in the dataset; handle missing values, convert the numerical columns in the data to numerical types, and fill the missing values in the numerical columns with the mean value; separate the feature data and label data from the training set and test set data; perform undersampling on the training set to ensure balance between classes; print the class distribution after undersampling; save the processed training set data and test set data (without undersampling) as new CSV files.

[0106] Step 3) Use Python to simulate the method proposed in this patent, set the initial parameter of the maximum number of iterations to 10, the final number of features to 30, and the initial weight of each method to The relevant parameters of each feature selection method are default parameters, the regularization parameter of the SVM model is 1, the smoothing coefficient of the contribution factor is 0.7, the fluctuation suppression intensity coefficient is 0.5, and the kernel function type is the radial basis function kernel for feature selection.

[0107] Step 4) Use the features finally selected by the method proposed in this patent to train an SVM model on the test set, perform the fault prediction task, and conduct a comparative experiment with the existing feature selection methods based on the filter method and the wrapper method. The experimental results are shown in Table 1 - Table 3:

[0108] Table 1 Evaluation Table of the Experimental Effect of This Method

[0109]

[0110] Table 2 Evaluation Table of the Experimental Effect of the Filter Method

[0111]

[0112] Table 3 Evaluation Table of the Experimental Effect of the Wrapper Method

[0113]

[0114] The experimental results show that an adaptive hybrid feature selection method for industrial big data proposed in this patent has certain superiority compared with the existing methods, has better fault prediction accuracy and precision, and has better versatility and flexible adjustment ability, and can be better applied to different high-dimensional data mining fields.

[0115] It should be noted that the above embodiments are not intended to limit the protection scope of the present invention. Any equivalent transformation or substitution made on the basis of the above technical solutions falls within the protection scope of the claims of the present invention.

Claims

1. An adaptive hybrid feature selection method for industrial big data, characterized in that: The method comprises the following steps: Firstly, a hybrid feature selection method is proposed, which uses multiple predefined feature selection methods to evaluate the feature set and combines the candidate feature subsets selected by each method to obtain the candidate feature set. Secondly, a dynamic evaluation method for feature selection is proposed to dynamically evaluate the performance contribution and performance volatility of each feature selection method during the iteration process; Then, an adaptive adjustment mechanism is proposed to adaptively adjust the participation degree of each method in the next round of iteration during the iteration process; Finally, repeat the above steps, add the candidate feature set obtained in each round to the final feature set, and stop the iteration when the final feature set reaches the predetermined size to achieve the final feature selection.

2. The adaptive hybrid feature selection method for industrial big data according to claim 1, characterized in that: A hybrid feature selection method, The following steps are involved: First, obtain the complete original feature set F = {f1,f2,…f n }, Then, the initial feature set F is evaluated independently by running the hybrid feature selection methods in parallel, and each method generates a subset of candidate features. Among them, the hybrid feature selection method includes a feature selection method M1 based on the Pearson correlation coefficient, and the calculation formula is as follows: Among them, x i ,y i are the values ​​of the i-th sample on feature X and target variable Y, respectively. are the means of all samples on feature X and target variable Y, n is the total number of samples, and the number of generated features is The candidate feature subset Among them, the hybrid feature selection method includes a feature selection method M2 based on information gain, and the calculation formula is as follows: IG(X,Y)=H(Y)-H(Y|X) Among them, H(Y) represents the uncertainty of the target variable Y, and the calculation formula is as follows: Among them, P(y) represents the probability that the target variable takes the value y. Among them, H(Y|X) represents the uncertainty of the target variable Y when the feature X is known, and the calculation formula is as follows: Among them, P(x) represents the probability that feature X takes the value x, P(y|x) represents the conditional probability that Y is y under the condition that X is x, and the number of generated features is The candidate feature subset Among them, the hybrid feature selection method includes the feature selection method M3 based on Lasso regularization, and the calculation formula of its objective function is as follows: Where N represents the total number of samples, y i represents the actual value of the i-th sample, represents the predicted value of the i-th sample, λ represents the regularization hyperparameter that controls the strength of the penalty term, and p represents the number of features. is the L1 norm, which represents the sum of the absolute values ​​of the feature coefficients, and the number of generated features is The candidate feature subset Finally, the candidate feature subsets generated by each feature selection method are collected to obtain the candidate feature set C for this round of iteration. t , and C t The final feature set S in the previous iteration t-1 Merge into the final feature set S of the current round t .

3. The adaptive hybrid feature selection method for industrial big data according to claim 1 is characterized in that: The feature selection dynamic evaluation method comprises the following steps: First, according to the currently obtained c j ∈C t , and combine it with the final feature set S in the previous iteration t-1 The feature set S′ is merged into t =S t ∪{c j }, Then, using the feature subset S′ t Train a support vector machine model, the calculation formula is as follows: f(x)=sign(W T x+b) Among them, x is the sample input vector, which contains the feature subset S′ t All the features in , w is the weight vector of the support vector machine, which represents the linear combination of features, b is the bias term, f(x) is the predicted output of the model, which represents the predicted value of sample x belonging to a certain category. The training goal of the support vector machine is to find a decision boundary that maximizes the interval, so that: Among them, y i is x i The true label, w and b are the parameters to be optimized, and the classification error rate Err(S′ t ), the calculation formula is as follows: Where n is the number of samples, is the predicted label, y i is the true label, It is an indicator function. When the prediction is wrong, the function value is 1, otherwise it is 0. Calculate the current feature set S t and contains candidate features c j The new feature set S′ t The error change is used to evaluate the contribution of the feature selection method, and the calculation formula is as follows: If the error is reduced (i.e. ), indicating that the method improves the performance of the model; otherwise, it indicates that the method does not improve the model much. Next, calculate the feature selection method M i Performance contribution index The calculation formula is as follows: Finally, the feature selection method M is calculated i Performance volatility index The calculation formula is as follows: in, It is method M i Performance contribution indicator in the last k iterations, It is French i The average performance contribution indicator in the most recent k iterations.

4. The adaptive hybrid feature selection method for industrial big data according to claim 1 is characterized in that: The adaptive adjustment mechanism comprises the following steps: First, select method M according to the acquired features i Performance contribution index and performance volatility indicators Introducing a dual-factor weight smoothing mechanism, through historical weight memory and dual parameter adjustment, to avoid weight mutations and calculate the participation weight of this method The calculation formula is as follows: Among them, β is the contribution factor smoothing coefficient, γ is the volatility suppression strength coefficient, represents the weight value of the previous iteration, Then, according to the feature selection method M i The weight of participation Introduce a dynamic candidate feature number allocation mechanism and adjust the method to generate a candidate feature set in the next iteration The number of features The calculation formula is as follows: Among them, N t is the total number of candidate features in this round of iteration, ceil is the round-up operation, when When making adjustments, the following priorities are followed: The method with the largest value; second priority to ensure the current round The method with the lowest mean, Finally, the candidate feature set C in the next iteration is adaptively adjusted through the above steps t The proportion of candidate features generated by each method in the ,flexibly adjusting the participation of each method.

5. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, an adaptive hybrid feature selection method for industrial big data as described in any one of claims 1 to 4 is implemented.

6. A computer-readable storage medium having computer instructions stored thereon, characterized in that: When the computer instruction is executed by a processor, an adaptive hybrid feature selection method for industrial big data as described in any one of claims 1 to 4 is implemented.

Images