A method for predicting the end-point quality of metallurgical processes based on a stacking ensemble strategy

Abstract

A method for predicting the endpoint quality of a metallurgical process based on a stacking integration strategy, collecting metallurgical process data; based on mechanism analysis and on-site expert experience, using random forest or PCA technology to perform feature screening to obtain key input variables; standardization, missing value, and outlier data cleaning and processing; dividing the data into a training set and a test set; using a six-fold cross-validation method, using parallel-arranged random forest regression, extreme random forest regression, and extreme gradient boosting to train multiple primary integration models and form the first network layer; using a Bayesian optimization algorithm to optimize and determine hyperparameters; constructing a second network layer by logistic linear regression, using the training set and test set integrated by the first network layer as data sets, performing meta-learner training, and finally obtaining the endpoint prediction output; using the test set data to evaluate the model performance; and performing prediction. This method can improve the accuracy of metallurgical endpoint quality prediction and the adaptability of the model.

Description

Technical Field

[0001] The invention belongs to the technical field of industrial production processes, and in particular relates to a method for predicting the end point quality of a metallurgical process based on a stacking integration strategy. Background Art

[0002] In the actual metal smelting process, in order to avoid environmental pollution, and to ensure a high temperature and high pressure reaction environment and make the reaction process easy to control, the metallurgical smelting process is often carried out in a closed reaction equipment. In this way, it is impossible to use sensors to directly measure the dynamic input and output variables of the metallurgical smelting process, nor is it possible to use mechanism analysis technology to model the dynamic characteristics of the reaction process. In the metallurgical process, sensors can only collect the initial set variables and end product quality data of the entire smelting process. This data is also called production report data. Report data is different from time series data, and it is impossible to use the existing time series network to predict trends, seasonality and periodicity. There is a complex nonlinear relationship between the input and output variables of the report data. It is difficult to fully explore the internal characteristics of the complex reaction process using existing conventional or simple nonlinear fitting functions and networks, and it is impossible to obtain satisfactory prediction results. Limited by the actual production process, it is often impossible to obtain sufficient and ideal report data. At the same time, since the collected report data often inevitably contains noise, singular values, invalid values, etc., it is necessary to use data cleaning technology to process the data. Therefore, using stacking integration technology to complement the advantages of existing data-driven modeling is an important practical need to achieve accurate prediction of metallurgical process report data.

[0003] In recent years, various data-driven modeling and prediction technologies have been proposed, such as identification modeling technology, polynomial fitting technology, neural network modeling technology, support vector machine technology, etc. However, conventional or simple-structured fitting functions and networks are difficult to fully explore the internal characteristics of complex reaction processes, and are unable to accurately model metallurgical process report data. At the same time, these algorithms require the collection of enough report data to obtain satisfactory prediction results.

[0004] Therefore, it is urgent to provide a method that can improve the accuracy of metallurgical process endpoint quality prediction based on production report data, so as to solve the problem of high-quality predictive modeling of metallurgical processes when process dynamic data cannot be obtained. Summary of the invention

[0005] In view of the problems existing in the above-mentioned prior art, the present invention provides a method for predicting the endpoint quality of a metallurgical process based on a stacking integration strategy. The method can improve the accuracy of the prediction of the metallurgical endpoint quality and the adaptability of the model. At the same time, it can effectively solve the problems of poor prediction effect and adaptability of traditional simple networks, and has practical significance for the prediction of metallurgical process report data.

[0006] In order to achieve the above object, the present invention provides a method for predicting the endpoint quality of a metallurgical process based on a stacking integration strategy, comprising the following steps:

[0007] Step 1: Collect metallurgical process data;

[0008] Use multiple sets of sensors installed at key collection points of the converter system to collect historical data of the converter smelting process;

[0009] Step 2: Based on mechanism analysis and on-site expert experience, random forest or PCA technology is used to perform feature screening to obtain key input variables;

[0010] Step 3: Standardization, missing value and outlier data cleaning and processing;

[0011] For the collected data, after cleaning and standardization, missing values ​​and outliers of the data as well as data normalization steps are realized;

[0012] Step 4: Divide the cleaned data into training set and test set;

[0013] Step 5: Use the six-fold cross validation method to train multiple primary ensemble models using parallel random forest regression, extreme random forest regression, and extreme gradient boosting to form the first network layer. The specific process is as follows:

[0014] S11: Building stacking framework;

[0015] A nonlinear integration strategy is used to combine heterogeneous learners to form a multi-layer learner with stronger learning ability, and the deep learner uses the output results of the primary learner as the data set to realize the re-learning function. The specific process is as follows:

[0016] A1: Divide the preprocessed data set into a training set and a test set;

[0017] A2: Select N primary learners for data training;

[0018] A3: The training set is further divided into K subsets of equal size for cross-validation;

[0019] A4: For each primary learner, use K-1 subsets as training data and the remaining 1 subset as validation data; each primary learner generates K model validation outputs, i.e. [v1,v2,…,v k ], these verification outputs are combined into a verification output set V1, and the verification output set [V1, V2, …, V N ];

[0020] A5: Input the test set into each trained primary learner to generate K groups of test outputs [t1, t2, …, t k ], and form the output set T1, then N primary learners obtain N test output sets [T1, T2, …, T N ];

[0021] A6: Merge the validation output sets and test output sets of all primary learners to form the training set and test set for the second-layer network training.

[0022] S12: Build a random forest regression model;

[0023] The regression tree is used as the base learner. The multiple base learners corresponding to N independent parallel regression trees are made independent of each other through the parallel combination integration strategy. The Bootstrap Aggregation (Bagging) integration method is used to form a strong learner. The target prediction value f(x) of the random forest regression algorithm that minimizes the objective function is obtained according to the Bagging construction algorithm in formula (1).

[0024]

[0025] In the formula, f n (x) is the prediction result obtained by each regression tree that does not affect each other's training; f is the model learned from the dataset D;

[0026] S13: Constructing extreme random forest regression models;

[0027] In the selection of training sets, all training samples are used to obtain each regression tree; in terms of node splitting, a random number between the maximum and minimum values ​​of the feature attribute is randomly selected. When the feature attribute value of the sample is greater than this value, it is used as the left branch, and when it is less than this value, it is used as the right branch; the purpose of randomly assigning samples to branches under the feature attribute is achieved; then the bifurcation value at this time is calculated; all feature attributes in the node are traversed, and the bifurcation values ​​of all feature attributes are obtained according to the above method; and the form with the largest bifurcation value is selected to achieve the bifurcation of the node;

[0028] S14: Constructing extreme gradient boosting model;

[0029] Through the Boosting integration idea, multiple regression tree models are linearly combined, and the Boosting construction algorithm in formula (2) is used to establish a prediction model with the minimum error through continuous iteration;

[0030]

[0031] In the formula, F m-1 (x) is the prediction result of the previous round, The forecast results for the current round;

[0032] Step 6: Use Bayesian optimization algorithm to optimize and determine hyperparameters;

[0033] The Bayesian optimization algorithm is used to determine the hyperparameters that have a greater impact on the calculation accuracy of the primary learner, so as to enhance the ability of the primary learner to learn data; the posterior probability distribution is constructed through the Bayesian optimization algorithm to predict the potential value of the objective function under different hyperparameter combinations, and this posterior probability distribution is used as an indicator of the hyperparameter search direction;

[0034] Step 7: Construct the second network layer by logistic linear regression, use the training set and test set integrated by the first network layer as the data set, and perform meta-learner training, so as to re-extract the prediction results of the first network layer as features, construct a two-layer stacking model, and finally obtain the endpoint prediction output;

[0035] Step 8: Use the test set data to evaluate the performance of the double-layer stacking model;

[0036] Step 9: Predict the converter endpoint temperature, Mn and S mass composition using the double-layer stack model.

[0037] In step three, missing values ​​in the collected data are screened and filled using interpolation, so that the model can be trained with more data without affecting the relationship between input and output features, making the model more accurate; statistical methods are used to detect abnormal values ​​and outliers in the data, and the converter data collected with outliers are excluded; the input features are standardized by formula (3) to unify the data into a relatively standard range, and at the same time, the weight imbalance problem caused by dimensional differences between features can be eliminated;

[0038]

[0039] In the formula, is the normalized value, x is the original value, μ is the sample mean, and σ is the sample standard deviation.

[0040] In S12 of step 1, the random forest regression model construction process is as follows:

[0041] B1: According to formula (4), k sample data are randomly selected with replacement from K initial data sets with equal probability to form set D;

[0042] D=(x1,Y1),(x2,Y2),…,(x k ,Y k ) (4);

[0043] In the formula, x i is the sample feature obtained by sampling; Yi is the corresponding sample label;

[0044] B2: Repeat B1 to obtain N groups of sampling sets, and train the corresponding regression tree based on each sampling set;

[0045] B3: In each regression tree, randomly select h attributes as an attribute set; calculate the variance reduction between the prediction result and the target value of each feature to measure the contribution of the feature to the model in each split, and use the feature with the largest variance reduction as the best split node to achieve regression tree construction;

[0046] B4: According to steps B1 to B4, N independent parallel regression trees are established to form a random forest regression model.

[0047] In S14 of step 1, the construction process of the extreme gradient boosting model is as follows:

[0048] C1: predict the objective function L(θ) of extreme gradient boosting according to formula (5);

[0049]

[0050] In the formula, n is the number of samples; y i is the true value of the i-th XGBoost regression sample; t is the number of regression trees; is the loss function; is the predicted value of t round iteration; is the predicted value of the t-1th iteration; f t (x i ) is the model prediction value of the tth iteration; Ω(f t ) is the penalty term for the model complexity of the t-th regression tree, i.e., the L1 regularization term, which is used to prevent overfitting. Where T is the number of leaf nodes in the regression tree; is the complexity parameter of the regression tree; λ is the L2 regularization coefficient; ω is the modulus of the leaf node vector;

[0051] C2: In the second-order Taylor expansion formula On the basis of Assume x, f t (x i ) is assumed to be Δx, then the formula (12) is transformed into Expressed as f(x+Δx), then It is expressed as f(x);

[0052] C3: Through the second-order Taylor expansion principle, the modified form of the objective function L(θ) is obtained using formula (6);

[0053]

[0054] In the formula, g i is the first-order derivative of the loss function; h i is the second-order derivative of the loss function;

[0055] C4: Separate the constant term and function term of the objective function in formula (8) to obtain formula (7);

[0056]

[0057] In the formula, I j ={i|q(x i )=j} is the value of regression tree q on leaf node j, that is, the sample value on the jth leaf node;

[0058] C5: Introducing Real And the regularization term L1, combined Rewrite the objective function L(θ) of the extreme gradient boosting regression tree using formula (8);

[0059]

[0060] In the formula, ω j is the weight of the leaf node;

[0061] C6: Derivative formula (8) and set it to 0, and obtain ω through formula (9) j Expression of

[0062]

[0063] C7: The feature partitioning of the regression tree using the information gain in formula (10) in the extreme gradient boosting algorithm;

[0064]

[0065] In the formula, is the optimal value of the objective function (L) partitioned into the left subtree; is the optimal value of the objective function (R) for partitioning to the right subtree; It is the objective function of the node before node splitting;

[0066] C8: Determine each regression tree based on the above information gain to construct an integrated model of extreme gradient boosting; for the generated regression tree, use the minimum loss function as the standard and use the validation data set to prune and determine the optimal subtree; the pruning technology merges some leaf nodes in the tree into new internal nodes to reduce the complexity of the model; a greedy algorithm is used for pruning; that is, at each node, extreme gradient boosting calculates the loss function of keeping the node as a leaf node or continuing to split the node, and selects the solution with the smallest loss function as the final solution for the node.

[0067] In step 4, the cleaned data is divided into training set and test set in a ratio of 7:3.

[0068] In step six, the optimization process of the Bayesian optimization algorithm is as follows:

[0069] D1: Based on Gaussian process regression, construct the posterior probability distribution p(J|E) according to formula (11);

[0070]

[0071] Where p(J|E) is the probability of the training data E appearing given J; p(J) is the prior probability of J; p(E) is the probability of the data E appearing;

[0072] D2: In the next iteration, a new hyperparameter vector u is selected n+1 , and use formula (12) to calculate its posterior probability p(J n+1 |E); Select the hyperparameter vector with the smallest expected objective function value And update the search direction; calculate the expected value Ε[J by solving the Gaussian integral in formula (4) n+1 |E];

[0073]

[0074] In the formula, μ n+1 is the mean;

[0075] D3: Through the iterative process and updated search direction, the objective function is continuously optimized until the optimal hyperparameter vector is obtained. Make the objective function minimum, where u is a d-dimensional vector containing d hyperparameters of the target search, i.e. u = (u1, u2, ..., u d ).

[0076] The present invention is a two-layer nonlinear process prediction network based on probability decision-making based on a stacking integration strategy. For the collected historical production data, preliminary feature screening is performed according to the system working mechanism and expert experience. The data is preprocessed and feature variables are selected, and the cleaned and standardized data are obtained. The data is divided into a training set and a test set. The first network layer is composed of a random forest regression (RFR), an extreme random forest regression (EFR) and an extreme gradient boosting (XGBoost) algorithm arranged in parallel, and the data is trained in combination with a six-fold cross-validation method and a Bayesian optimization algorithm. The first layer uses two integrated algorithms RFR and XGBoost as primary learners, and by setting different random seed numbers and model parameters, it is ensured that the model can capture the diversity characteristics between data, and the generalization ability of the overall model is also improved. The single RFR is an integrated model composed of regression trees that use a binary method to divide nodes, which cannot capture the nonlinear relationship between complex data, resulting in deviations in the prediction results. Stacking multiple RFR structures can correct the prediction deviation of a single RFR through the meta-learner of the second network layer to improve the prediction accuracy of the model. Since EFR randomly selects the division points of the feature values, the scale of the regression tree generated by it is larger than that of the regression tree generated by RFR. EFR can effectively reduce the prediction variance. When constructing the decision tree, the features, parameters, models and splits of EFR are all selected in a random manner. Therefore, EFR can improve the capture of data feature diversity by the stacked network. Since the internal decision tree of XGBoost adopts a parallel operation mode, multiple stacked XGBoost structures can significantly improve the training speed of the proposed network. In addition, the regularization term of each XGBoost model can also reduce the risk of overfitting of a single model and improve the generalization ability of the overall network. During model training, the Bayesian optimization algorithm is used to optimize and determine the hyperparameters of the first-layer integrated model. The second network layer uses the output results of the trained first network layer as features, and obtains the endpoint prediction output by logistic linear regression. The use of logistic linear regression in the second layer can simplify the proposed model and reduce the risk of overfitting of the model. The stacking architecture can make the advantages and disadvantages of each model complement each other and improve the learning ability of the double-layer stacked network.

[0077] The present invention uses stacking integration technology to predict the endpoint quality of the metallurgical process. By integrating RFR, EFR and XGBoost) algorithms side by side in the first layer to ensure the generalization ability of the model and capture the diversity characteristics between data, the endpoint prediction output is obtained by logistic linear regression in the second network layer. This strategy makes the advantages and disadvantages of each model complement each other, effectively improves the accuracy of the double-layer stacking network for the prediction of metallurgical endpoint quality and the adaptability of the model, improves the accuracy of the prediction of the endpoint temperature, manganese (Mn) and sulfur (S) content in the actual converter production process, and effectively solves the problem of poor prediction effect and adaptability of the traditional simple network. The double-layer prediction network based on the stacking integration strategy provided by the present invention has practical significance for the prediction of metallurgical process report data. BRIEF DESCRIPTION OF THE DRAWINGS

[0078] Figure 1 The quality prediction flow chart of the present invention based on the double-layer stacked integrated network;

[0079] Figure 2 A flowchart of the Bayesian optimization algorithm in the present invention;

[0080] Figure 3 Output prediction results for different networks for the test data of temperature (y1);

[0081] Figure 4 Output prediction results for different networks on the test data of manganese (y2);

[0082] Figure 5 Output prediction results for different networks for the test data of sulfur (y3);

[0083] Figure 6 Histogram of prediction errors for different networks. DETAILED DESCRIPTION

[0084] The present invention will be further described below in conjunction with the embodiments.

[0085] like Figures 1 to 6 As shown, the present invention provides a method for predicting the end point quality of a metallurgical process based on a stacking integration strategy, comprising the following steps:

[0086] Step 1: Collect metallurgical process data;

[0087] Use multiple sets of sensors installed at key collection points of the converter system to collect historical data of the converter smelting process;

[0088] Step 2: Based on mechanism analysis and on-site expert experience, random forest or PCA technology is used to perform feature screening to obtain key input variables;

[0089] Step 3: Standardization, missing value and outlier data cleaning and processing;

[0090] For the collected data, after cleaning and standardization, missing values ​​and outliers of the data as well as data normalization steps are realized;

[0091] The collected metallurgical process data may have problems such as recording errors and loss, so it is necessary to remove singular values ​​and null values ​​from the collected data set. Normalization technology can be used to process the collected data to solve the problem that the model is too sensitive to local features due to large differences in the values ​​of features, ensuring that the influence of each feature on the model is balanced to improve the efficiency of model training.

[0092] Missing values ​​in the collected data are screened and filled using interpolation methods, so that the model can be trained with more data without affecting the relationship between input and output features, making the model more accurate; using statistical methods to detect abnormal values ​​and outliers in the data, and excluding the converter data collected with outliers; using formula (3) to standardize the input features, unify the data into a relatively standard range, and eliminate the weight imbalance problem caused by dimensional differences between features;

[0093]

[0094] In the formula, is the normalized value, x is the original value, μ is the sample mean, and σ is the sample standard deviation.

[0095] Step 4: Divide the cleaned data into training set and test set;

[0096] The cleaned data is preferably divided into a training set and a test set in a ratio of 7:3.

[0097] Step 5: Use the six-fold cross-validation method to train multiple primary ensemble models using parallel-arranged random forest regression (RFR), extreme random forest regression (EFR) and extreme gradient boosting (XGBoost) to form the first network layer. The specific process is as follows:

[0098] S11: Building stacking framework;

[0099] A nonlinear integration strategy is used to combine heterogeneous learners to form a multi-layer learner with stronger learning ability, and the deep learner uses the output results of the primary learner as the data set to realize the re-learning function. The specific process is as follows:

[0100] A1: Divide the preprocessed data set into a training set and a test set;

[0101] A2: Select N primary learners for data training;

[0102] A3: The training set is further divided into K subsets of equal size for cross-validation;

[0103] A4: For each primary learner, use K-1 subsets as training data and the remaining 1 subset as validation data; each primary learner generates K model validation outputs, i.e. [v1,v2,…,v k ], these verification outputs are combined into a verification output set V1, and the verification output set [V1, V2, …, V N ];

[0104] A5: Input the test set into each trained primary learner to generate K groups of test outputs [t1, t2, …, t k ], and form the output set T1, then N primary learners obtain N test output sets [T1, T2, …, T N ];

[0105] A6: Merge the validation output sets and test output sets of all primary learners to form the training set and test set for the second-layer network training.

[0106] S12: Build a random forest regression model;

[0107] The random forest regression algorithm uses regression trees as base learners. Through the parallel combination integration strategy, multiple base learners corresponding to N independent parallel regression trees are made independent of each other. The Bootstrap Aggregation (Bagging) integration method is used to form a strong learner. According to the Bagging construction algorithm in formula (1), the target prediction value f(x) of the random forest regression algorithm that minimizes the objective function is obtained.

[0108]

[0109] In the formula, f n (x) is the prediction result obtained by each regression tree that does not affect each other's training; f is the model learned from the dataset D;

[0110] Specifically, the random forest regression model construction process is as follows:

[0111] B1: According to formula (4), k sample data are randomly selected with replacement from K initial data sets with equal probability to form set D;

[0112] D=(x1,Y1),(x2,Y2),…,(x k ,Y k ) (4);

[0113] In the formula, x iis the sample feature obtained by sampling; Y i is the corresponding sample label;

[0114] B2: Repeat B1 to obtain N groups of sampling sets, and train the corresponding regression tree based on each sampling set;

[0115] B3: In each regression tree, randomly select h attributes as an attribute set; calculate the variance reduction between the prediction result and the target value of each feature to measure the contribution of the feature to the model in each split, and use the feature with the largest variance reduction as the best split node to achieve regression tree construction;

[0116] B4: According to steps B1 to B4, N independent parallel regression trees are established to form a random forest regression model.

[0117] Considering that bagging is a method of training models independently on different sampling sets, it reduces the variance of model predictions, thereby preventing overfitting. In addition, since the bagging algorithm samples each time to train the model, it has strong generalization ability. The probability of each sample being randomly sampled is equal. When k tends to infinity, the probability convergence domain is shown in formula (13);

[0118]

[0119] Formula (13) indicates that 63.2% of the dataset is randomly divided into training and validation, while the remaining 36.8% of the training data is not involved in modeling, that is, out-of-bag data, which can be used to evaluate the generalization ability of the model. In addition, the generalization error of random forest regression is As shown in formula (14);

[0120]

[0121] in, The derivation process is as follows:

[0122] Define f(x) according to formula (15);

[0123] f(x)=E D [f(x;D)] (15);

[0124] Where f(x; D) is the predicted output of model f for x learned from dataset D.

[0125] According to formula (16), the bias is defined as 2 (x), bias 2 (x) is the difference between the predicted value and the true value, which can measure the accuracy of the model;

[0126] bias2 (x) = (f(x) - y) 2 (16);

[0127] Where y is the true value of x.

[0128] According to formula (17), the variance of the results output by the model on different data sets is defined. The variance is used to measure the stability of the model.

[0129] variance=var(x)=E D [(f(x;D)-f(x)) 2 ] (17);

[0130] According to formula (18), the noise that is inevitable in the data collection process and has nothing to do with the actual distribution of the data is defined as 2 ;

[0131] noise 2 =E D [(y D -y) 2 ] (18);

[0132] In the formula, y D is the target value of x in the dataset D.

[0133] According to formula (19), it is assumed that the expectation of the noise is 0;

[0134] E D [y D -y] = 0 (19);

[0135] The simplified process is shown in formula (20);

[0136]

[0137] The random forest structure can handle very high dimensional data without additional dimensionality reduction operations, and uses parallelization methods to speed up the training of the model. In addition, random forests have good noise resistance and are not prone to overfitting.

[0138] S13: Constructing extreme random forest regression models;

[0139] Extreme random forest regression is also an integrated method based on regression trees. In terms of training set selection, EFR uses all training samples to obtain each regression tree; in terms of node splitting, it randomly selects an arbitrary number between the maximum and minimum values ​​of the feature attribute. When the feature attribute value of the sample is greater than this value, it is used as the left branch, and when it is less than this value, it is used as the right branch; the purpose of randomly assigning samples to branches under this feature attribute is achieved; then the bifurcation value at this time is calculated; all feature attributes in the node are traversed, and the bifurcation values ​​of all feature attributes are obtained according to the above method; and the form with the largest bifurcation value is selected to realize the bifurcation of the node; the construction method of EFR determines its strong robustness to noise and outliers and its characteristics of not being easy to overfit.

[0140] S14: Constructing extreme gradient boosting model;

[0141] XGBoost uses the Boosting integration concept to linearly combine multiple regression tree models, and uses the Boosting construction algorithm in formula (2) to establish a prediction model with the minimum error through continuous iteration;

[0142]

[0143] In the formula, F m-1 (x) is the prediction result of the previous round, The forecast results for the current round;

[0144] The premise of the Boosting construction algorithm is that the added regression trees are to make the model prediction better, that is, the loss decreases after adding new regression trees. The construction process of the extreme gradient boosting model is as follows:

[0145] C1: predict the objective function L(θ) of extreme gradient boosting according to formula (5);

[0146]

[0147] In the formula, n is the number of samples; y i is the true value of the i-th XGBoost regression sample; t is the number of regression trees; is the loss function; is the predicted value of t round iteration; is the predicted value of the t-1th iteration; f t (x i ) is the model prediction value of the tth iteration; Ω(f t ) is the penalty term for the model complexity of the t-th regression tree, i.e., the L1 regularization term, which is used to prevent overfitting. Where T is the number of leaf nodes in the regression tree; is the complexity parameter of the regression tree; λ is the L2 regularization coefficient; ω is the modulus of the leaf node vector;

[0148] C2: In the second-order Taylor expansion formula On the basis of Assume x, f t (x i ) is assumed to be Δx, then the formula (12) is transformed into Expressed as f(x+Δx), then It is expressed as f(x);

[0149] C3: Through the second-order Taylor expansion principle, the modified form of the objective function L(θ) is obtained using formula (6);

[0150]

[0151] In the formula, g i is the first-order derivative of the loss function; h i is the second-order derivative of the loss function;

[0152] C4: Separate the constant term and function term of the objective function in formula (8) to obtain formula (7);

[0153]

[0154] In the formula, I j ={i|q(x i )=j} is the value of regression tree q on leaf node j, that is, the sample value on the jth leaf node;

[0155] C5: In the optimization process, there is no need to consider the constant term of the constant objective function; introduce the real And the regularization term L1, combined Rewrite the objective function L(θ) of the extreme gradient boosting regression tree using formula (8);

[0156]

[0157] In the formula, ω j is the weight of the leaf node;

[0158] C6: Derivative formula (8) and set it to 0, and obtain ω through formula (9) j Expression of

[0159]

[0160] C7: The feature partitioning of the regression tree using the information gain in formula (10) in the extreme gradient boosting algorithm;

[0161]

[0162] In the formula, is the optimal value of the objective function (L) partitioned into the left subtree; is the optimal value of the objective function (R) for partitioning to the right subtree; It is the objective function of the node before node splitting;

[0163] C8: Determine each regression tree based on the above information gain to construct an integrated model of extreme gradient boosting; for the generated regression tree, use the minimum loss function as the standard and use the validation data set to prune and determine the optimal subtree; the pruning technology merges some leaf nodes in the tree into new internal nodes to reduce the complexity of the model; a greedy algorithm is used for pruning; that is, at each node, extreme gradient boosting calculates the loss function of keeping the node as a leaf node or continuing to split the node, and selects the solution with the smallest loss function as the final solution for the node.

[0164] Step 6: Use Bayesian optimization algorithm to optimize and determine hyperparameters;

[0165] The Bayesian optimization algorithm is used to determine the hyperparameters that have a greater impact on the calculation accuracy of the primary learner, so as to enhance the ability of the primary learner to learn data; the posterior probability distribution is constructed through the Bayesian optimization algorithm to predict the potential value of the objective function under different hyperparameter combinations, and this posterior probability distribution is used as an indicator of the hyperparameter search direction;

[0166] The optimization process of the Bayesian optimization algorithm is as follows:

[0167] D1: Based on Gaussian process regression, construct the posterior probability distribution p(J|E) according to formula (11);

[0168]

[0169] Where p(J|E) is the probability of the training data E appearing given J; p(J) is the prior probability of J; p(E) is the probability of the data E appearing;

[0170] D2: In the next iteration, a new hyperparameter vector u is selected n+1 , and use formula (12) to calculate its posterior probability p(J n+1 |E); Select the hyperparameter vector with the smallest expected objective function value And update the search direction; calculate the expected value Ε[J by solving the Gaussian integral in formula (4) n+1 |E];

[0171]

[0172] In the formula, μ n+1 is the mean;

[0173] D3: Through the iterative process and updated search direction, the objective function is continuously optimized until the optimal hyperparameter vector is obtained. Make the objective function minimum, where u is a d-dimensional vector containing d hyperparameters of the target search, i.e. u = (u1, u2, ..., u d ).

[0174] Step 7: Construct the second network layer by logistic linear regression, use the training set and test set integrated by the first network layer as the data set, and perform meta-learner training, so as to re-extract the prediction results of the first network layer as features, construct a two-layer stacking model, and finally obtain the endpoint prediction output;

[0175] Step 8: Use the test set data to evaluate the performance of the double-layer stacking model;

[0176] Step 9: Predict the converter endpoint temperature, Mn and S mass composition using the double-layer stack model.

[0177] Experimental analysis:

[0178] The present invention takes the converter system for smelting Q195 steel as the object, and collects the historical data of the converter smelting process according to the steel metallurgical report of the factory (the historical data is collected by multiple groups of sensors installed on the key collection points of the converter system). The data of the converter report recording different furnace numbers and furnace ages are summarized and sorted. The historical data include: C, Si, Mn, P, S, molten iron volume, preheated scrap steel, steel bar cutting head, 12# scrap steel, 19# scrap steel, oxygen supply time, smelting cycle, oxygen consumption, lime, dolomite, limestone, magnesium balls, lump ore, solid waste balls, pellet return ore, steel sand aluminum, silicon manganese, ferrosilicon, post-furnace temperature measurement and other information about converter charging, temperature, smelting process time, operation and steel tapping. According to the guidance experience of experts, the molten iron volume and preheated scrap steel are added to obtain the input feature, molten iron volume. The input feature "total scrap steel" is obtained by adding "rebar cutting head", "12# scrap steel" and "19# scrap steel" in equal proportions; since "oxygen supply time", "smelting cycle" and "oxygen consumption" are all variables with serious linear correlation with the supply time, the most representative "oxygen consumption" is taken as the input feature. The features closely related to the steel-making situation, such as the temperature measurement after the furnace, whether to replenish the gun, the temperature of the argon station, the condition of the ladle, the steel flow time and the number of steel-making ports in the actual converter smelting process, and the features closely related to the iron addition time and the blowing time and the smelting process time are discarded as irrelevant variables as the features with little correlation with the temperature and the end-point quality prediction of Mn and S. Most of the auxiliary materials such as lime, dolomite, limestone, lump ore and the C, Si, Mn, P, and S components in the molten iron at the initial stage of steelmaking are retained as input features.

[0179] For the collected data, after cleaning and standardization, the missing values, outliers and data normalization steps are realized. Missing values ​​in the collected data are screened and filled using interpolation, so that the model can be trained with more data without affecting the relationship between the input and output features, making the model more accurate; using statistical methods such as box plots to detect outliers and outliers in the data, and excluding the converter data collected with outliers. Standardize the input features to unify the data into a relatively standard range, and at the same time eliminate the weight imbalance problem caused by dimensional differences between features. The standardization formula is as follows: In the formula, is the normalized value, x is the original value, μ is the sample mean, and σ is the sample standard deviation.

[0180] According to the guidance of expert experience, the input features were obtained after feature fusion and feature screening. 19 input operation variables and 3 key output variables were screened. A total of 5,000 sets of production report data were obtained. The three outputs are the mass components of the endpoint temperature y1, manganese y2, and sulfur y3. The data was divided into a training set and a test set in a ratio of 7:3, and the training set was divided again using a six-fold cross-validation method. The Bayesian optimization algorithm was used to optimize the hyperparameters of each primary model, so as to train multiple integrated primary models and obtain the first network layer of the proposed double-layer stacking model based on probability decision-making. The constructed second network layer re-extracts the prediction results of the first network layer as features, and finally obtains the endpoint prediction output. The test set data evaluates the performance of the double-layer stacking model and analyzes the accuracy of the model's prediction of the converter endpoint temperature, Mn and S mass components. The root mean square error (RMSE) is used to reflect the deviation between the predicted result and the true value. Where n is the number of samples, is the model prediction output, y i is the real sampling output; mean absolute error (MAE), Used to measure the accuracy of model prediction; R 2 ), Used to measure the degree of fit of the model to the data; Explanatory variance (EV), It is used to measure the degree to which the model explains the data and to quantitatively evaluate the predictive effects of different models.

[0181] The existing back propagation neural network (BP) and support vector regression (SVR) were also compared and verified. The prediction results of different networks for the three key indicators are shown in Table 1, Table 2 and Table 3. It can be seen that the RMSE, MAE, R 2 In order to show the results of different models predicting test data, the prediction results of three key output variables are plotted with different algorithm prediction values ​​and true values ​​as the vertical axis, as shown in the figure. Figure 3 , Figure 4 and Figure 5 In order to show the relative error results of different model prediction data, the relative error between the predicted value and the true value of different algorithms is used as the vertical axis to draw the cumulative average error bar chart results of the relative error of the prediction of the three key output variables, as shown in Figure 6 As shown. It can be seen that the proposed network is superior to other networks in terms of dynamic prediction results and numerical measurement. The present invention studies tabular data, and the actual converter input-output relationship presents complex nonlinear characteristics. Due to insufficient data sets and poor BP neural network training effects, simple BP networks are prone to fall into local optimal values. Since supported regression SVR is sensitive to measurement noise data and will be affected by outliers, it is difficult to adapt to converter smelting data collected on-site. The present invention accurately predicts the temperature, Mn and S mass components of the converter endpoint.

[0182] In summary, the invention provides a method for constructing a double-layer stacking model based on probability decision making, which realizes accurate prediction of converter end temperature, Mn and S quality. The invention effectively solves the problem of poor prediction effect and adaptability of traditional simple network. Accurate prediction of converter end quality facilitates control of converter end quality and improves product quality of steel.

[0183] Table 1. Prediction error of temperature (y1) by different networks

[0184]

[0185] Table 2. Prediction errors of different networks for manganese (y2)

[0186]

[0187] Table 3. Prediction errors of sulfur (y3) by different networks

[0188]

[0189] The present invention is a two-layer nonlinear process prediction network based on probability decision-making based on a stacking integration strategy. For the collected historical production data, preliminary feature screening is performed according to the system working mechanism and expert experience. The data is preprocessed and feature variables are selected, and the cleaned and standardized data are obtained. The data is divided into a training set and a test set. The first network layer is composed of a random forest regression (RFR), an extreme random forest regression (EFR) and an extreme gradient boosting (XGBoost) algorithm arranged in parallel, and the data is trained in combination with a six-fold cross-validation method and a Bayesian optimization algorithm. The first layer uses two integrated algorithms RFR and XGBoost as primary learners, and by setting different random seed numbers and model parameters, it is ensured that the model can capture the diversity characteristics between data, and the generalization ability of the overall model is also improved. The single RFR is an integrated model composed of regression trees that use a binary method to divide nodes, which cannot capture the nonlinear relationship between complex data, resulting in deviations in the prediction results. Stacking multiple RFR structures can correct the prediction deviation of a single RFR through the meta-learner of the second network layer to improve the prediction accuracy of the model. Since EFR randomly selects the division points of the feature values, the scale of the regression tree generated by it is larger than that of the regression tree generated by RFR. EFR can effectively reduce the prediction variance. When constructing the decision tree, the features, parameters, models and splits of EFR are all selected in a random manner. Therefore, EFR can improve the capture of data feature diversity by the stacked network. Since the internal decision tree of XGBoost adopts a parallel operation mode, multiple stacked XGBoost structures can significantly improve the training speed of the proposed network. In addition, the regularization term of each XGBoost model can also reduce the risk of overfitting of a single model and improve the generalization ability of the overall network. During model training, the Bayesian optimization algorithm is used to optimize and determine the hyperparameters of the first-layer integrated model. The second network layer uses the output results of the trained first network layer as features, and obtains the endpoint prediction output by logistic linear regression. The use of logistic linear regression in the second layer can simplify the proposed model and reduce the risk of overfitting of the model. The stacking architecture can make the advantages and disadvantages of each model complement each other and improve the learning ability of the double-layer stacked network.

[0190] The present invention uses stacking integration technology to predict the endpoint quality of the metallurgical process. By integrating RFR, EFR and XGBoost algorithms side by side in the first layer to ensure the generalization ability of the model and capture the diversity characteristics between data, the endpoint prediction output is obtained by logistic linear regression in the second network layer. This strategy makes the advantages and disadvantages of each model complement each other, effectively improves the accuracy of the double-layer stacking network for the prediction of metallurgical endpoint quality and the adaptability of the model, improves the accuracy of the prediction of the endpoint temperature, manganese (Mn) and sulfur (S) content in the actual converter production process, and effectively solves the problem of poor prediction effect and adaptability of the traditional simple network. The double-layer prediction network based on the stacking integration strategy provided by the present invention has practical significance for the prediction of metallurgical process report data.

Claims

1. A method for predicting the end point quality of a metallurgical process based on a stacking integration strategy, characterized in that: The following steps are involved: Step 1: Collect metallurgical process data; Use multiple sets of sensors installed at key collection points of the converter system to collect historical data of the converter smelting process; Step 2: Based on mechanism analysis and on-site expert experience, random forest or PCA technology is used to perform feature screening to obtain key input variables; Step 3: Standardization, missing value and outlier data cleaning and processing; For the collected data, after cleaning and standardization, missing values ​​and outliers of the data as well as data normalization steps are realized; Step 4: Divide the cleaned data into training set and test set; Step 5: Use the six-fold cross validation method to train multiple primary ensemble models using parallel random forest regression, extreme random forest regression, and extreme gradient boosting to form the first network layer. The specific process is as follows: S11: Building stacking framework; A nonlinear integration strategy is used to combine heterogeneous learners to form a multi-layer learner with stronger learning ability, and the deep learner uses the output results of the primary learner as the data set to realize the re-learning function. The specific process is as follows: A1: Divide the preprocessed data set into a training set and a test set; A2: Select A primary learner for data training; A3: The training set is further divided into Subsets of equal size were used for cross-validation; A4: For each primary learner, use A subset of them is used as training data, and the remaining one is used as verification data; each primary learner generates The model validation output is , these verification outputs are combined into the verification output set ,right The primary learner gets the verification output set ; A5: Input the test set into each trained primary learner to generate Group Test Output , and constitute the output set ,but The primary learner gets Test output collection ; A6: Merge the validation output sets and test output sets of all primary learners to form the training set and test set for the second-layer network training. S12: Build a random forest regression model; The random forest regression model construction process is as follows: B1: According to formula (4), random sampling with replacement is used to select The initial data set is extracted with medium probability The sample data is composed of a set ; （4）； In the formula, is the sample characteristic obtained by sampling; is the corresponding sample label; B2: Repeat the operation of B1 to obtain Group sampling sets, and train corresponding regression trees based on each sampling set; B3: In each regression tree, randomly select The attributes are taken as an attribute set; the variance reduction between the prediction result and the target value of each feature is calculated to measure the contribution of the feature to the model in each segmentation, and the feature with the largest variance reduction is used as the best splitting node to realize the regression tree construction; B4: Follow steps B1 to B4 to create Independent parallel regression trees form a random forest regression model; The regression tree is used as the base learner, and the integration strategy of parallel combination is used to The multiple base learners corresponding to the independent parallel regression trees are independent of each other, and the Bootstrap Aggregation integration method is used to form a strong learner. According to the Bagging construction algorithm in formula (1), the target prediction value of the random forest regression algorithm that minimizes the objective function is obtained. ; （1）； In the formula, The prediction results obtained for each regression tree that does not affect each other's training; The data set The learned model; S13: Constructing extreme random forest regression models; In terms of training set selection, all training samples are used to obtain each regression tree; in terms of node splitting, a random number between the maximum and minimum values ​​of the feature attribute is selected. When the feature attribute value of the sample is greater than this value, it is used as the left branch, and when it is less than this value, it is used as the right branch. To achieve the purpose of randomly assigning samples to branches under the characteristic attribute; then calculate the bifurcation value at this time; traverse all characteristic attributes in the node, and obtain the bifurcation values ​​of all characteristic attributes according to the above method; and select the form with the largest bifurcation value to achieve the bifurcation of the node; S14: Constructing extreme gradient boosting model; The Boosting integration concept is used to linearly combine multiple regression tree models, and the Boosting construction algorithm in formula (2) is used to establish a prediction model with the minimum error through continuous iteration. （2）； In the formula, is the prediction result of the previous round, The forecast results for the current round; Step 6: Use Bayesian optimization algorithm to optimize and determine hyperparameters; The Bayesian optimization algorithm is used to determine the hyperparameters that have a greater impact on the calculation accuracy of the primary learner, so as to enhance the ability of the primary learner to learn data; the posterior probability distribution is constructed through the Bayesian optimization algorithm to predict the potential value of the objective function under different hyperparameter combinations, and this posterior probability distribution is used as an indicator of the hyperparameter search direction; Step 7: Construct the second network layer by logistic linear regression, use the training set and test set integrated by the first network layer as the data set, and perform meta-learner training, so as to re-extract the prediction results of the first network layer as features, construct a two-layer stacking model, and finally obtain the endpoint prediction output; Step 8: Use the test set data to evaluate the performance of the double-layer stacking model; Step 9: Predict the converter endpoint temperature, Mn and S mass composition through the double-layer stack model.

2. A method for predicting the end point quality of a metallurgical process based on a stacking integration strategy according to claim 1, characterized in that: In step three, missing values ​​in the collected data are screened and filled using interpolation methods, so that the model can be trained with more data without affecting the relationship between input and output features, making the model more accurate; statistical methods are used to detect abnormal values ​​and outliers in the data, and the converter data collected with outliers are excluded; Formula (3) is used to standardize the input features, unify the data into a relatively standard range, and eliminate the weight imbalance problem caused by dimensional differences between features. （3）； In the formula, is the normalized value, is the original value, is the sample mean, is the sample standard deviation.

3. The method for predicting the end point quality of a metallurgical process based on a stacking integration strategy according to claim 2, characterized in that: In S14 of step 1, the construction process of the extreme gradient boosting model is as follows: C1: The objective function of extreme gradient boosting predicted by formula (5) ; （5）； In the formula, ; is the number of samples; For the The true value of the XGBoost regression sample; is the number of regression trees; is the loss function; for The predicted value of the round iteration; for The predicted value of the round iteration; It is The model prediction value of the round iteration; For the The penalty term for the model complexity of the regression tree is The regularization term, which is used to prevent overfitting, ,in, is the number of leaf nodes in the regression tree; is the complexity parameter of the regression tree; for Regularization coefficient; is the modulus of the leaf node vector; C2: In the second-order Taylor expansion formula On the basis of Assume that , Assume that , then the transformed formula (12) is Expressed as ,So Expressed as ; C3: Using the second-order Taylor expansion principle, the objective function is obtained using formula (6): Variations of （6）； In the formula, , , is the first-order derivative of the loss function; is the second-order derivative of the loss function; C4: Separate the constant term and function term of the objective function in formula (8) to obtain formula (7); （7）； In the formula, , , For regression tree At the leaf node The value of Sample values ​​on leaf nodes; C5: Introducing Real And the regularization term L1, combined , using formula (8) to rewrite the objective function of the extreme gradient boosting regression tree ; （8）； In the formula, is the weight of the leaf node; C6: Derivative formula (8) and set it to 0, and then use formula (9) to get Expression of （9）； C7: The feature partitioning of the regression tree using the information gain in formula (10) in the extreme gradient boosting algorithm; （10）； In the formula, is the objective function for partitioning into the left subtree The optimal value of is the objective function for partitioning into the right subtree The optimal value of It is the objective function of the node before node splitting; C8: Determine each regression tree based on the above information gain to construct an integrated model of extreme gradient boosting; for the generated regression tree, use the minimum loss function as the standard and use the validation data set to prune and determine the optimal subtree; the pruning technology merges some leaf nodes in the tree into new internal nodes to reduce the complexity of the model; a greedy algorithm is used for pruning; that is, at each node, extreme gradient boosting calculates the loss function of keeping the node as a leaf node or continuing to split the node, and selects the solution with the smallest loss function as the final solution for the node.

4. The method for predicting the end point quality of a metallurgical process based on a stacking integration strategy according to claim 3, characterized in that: In step 4, the cleaned data is divided into training set and test set in a ratio of 7:

3.

5. The method for predicting the end point quality of a metallurgical process based on a stacking integration strategy according to claim 4, characterized in that: In step six, the optimization process of the Bayesian optimization algorithm is as follows: D1: Based on Gaussian process regression, construct the posterior probability distribution according to formula (11) ; （11）； In the formula, For a given In the case of Probability of occurrence; for The prior probability of For data Probability of occurrence; D2: In the next iteration, a new hyperparameter vector is selected , and use formula (12) to calculate its posterior probability ; Select the hyperparameter vector with the smallest expected objective function value , and update the search direction; calculate the expected value by solving the Gaussian integral in formula (4) ; （12）； In the formula, is the mean; D3: Through the iterative process and updated search direction, the objective function is continuously optimized until the optimal hyperparameter vector is obtained. ; Make the objective function The minimum, among which, is a dimensional vector, containing The hyperparameters of the target search are .

Images