Stacked network model-based sparse small sample industrial process quality prediction method

Abstract

The invention provides a method for predicting the quality of a sparse small sample industrial process based on a stacked network model. The method comprises the following steps: collecting end point quality report data of an industrial production process; performing hierarchical processing on the acquired data according to the missing rate, and removing abnormal data in combination with a quartile method and production experience; generating a synthetic data expansion small sample data set by adopting a conditional generative adversarial network; obtaining a first-layer basic model based on an accumulated contribution rate screening method of an SHAP value; constructing a first layer of a stacked integrated learning model and adjusting hyper-parameters by using Bayesian optimization; constructing a Ridge meta learning device to integrate the output of the basic model and constructing a second-layer network; a six-fold cross validation training model is adopted; predicting performance through a multi-index quantitative model based on the test set; and verifying the prediction precision of the end-point phosphorus content and the temperature by using real converter production data. The method can realize high-precision prediction of the end point quality index of the complex industrial generation process, and is beneficial to ensuring the product quality and improving the production efficiency.

Description

Technical Field

[0001] The present invention belongs to the technical field of intelligent control of industrial processes, and specifically relates to a method for predicting the quality of sparse small-sample industrial processes based on a stacked network model. Background Art

[0002] Accurately predicting the endpoint quality of closed, high-temperature, and high-pressure production processes such as chemical synthesis, nonferrous metal smelting, and iron and steelmaking is crucial for ensuring product quality and improving production efficiency. Industrial production processes often involve complex multiphase reactions and nonlinear coupling factors, making them difficult to model using mechanistic analysis techniques. Due to factors such as corrosion and harsh environments, existing detection technologies and equipment cannot capture comprehensive, dynamic production process data in real time. For closed industrial processes, only pre-production component variables and offline measurements of the final product are available. This type of data, encompassing only pre- and post-production data, is also known as production report data. Due to varying process standards and requirements, production report data is often sparsely distributed, meaning some variables are only available during certain production steps. When producing new product types and varieties, it is often difficult to collect sufficient data. Small sample sizes and the inevitable presence of anomalies complicate data mining and quality prediction. With the advancement of intelligent technologies, intelligent networks can be considered for data-driven modeling and quality prediction of industrial processes with sparse, small sample sizes. Traditional single networks have their advantages as well as inevitable disadvantages. Therefore, it is possible to consider using stacking technology to integrate different types of basic models to give full play to the advantages of different models, and by complementing each other's strengths and weaknesses, improve the overall model's prediction robustness and accuracy for the endpoint quality indicators of complex industrial processes.

[0003] In stacked ensemble learning models, the choice of base models directly impacts predictive performance, but existing selection methods have significant drawbacks. Heuristic selection strategies rely on the designer's experience and lack a systematic quantitative basis, potentially leading to a lack of diversity and complementarity in the model combination. Screening base models based on a single performance metric can overlook synergies between models, causing the constructed ensemble model to lose key information. While selecting base models through error correlation analysis can improve model diversity, it carries high computational complexity, making it difficult for the model to converge effectively in high-dimensional data scenarios, hindering the full potential of ensemble learning techniques. For small sample sizes of industrial production process data, using data generation techniques to generate synthetic data is key to improving model performance. Bootstrap resampling techniques, constrained by the original data distribution, result in a lack of diversity in the generated samples and difficulty capturing the complex patterns of the original data. Variational autoencoder data generation techniques, constrained by the assumption that the original data follows a Gaussian distribution, result in low-quality generated samples and difficulty adapting to high-dimensional nonlinear data. Basic generative adversarial network data generation techniques, however, suffer from training instability and poor adaptability to high-dimensional heterogeneous data, resulting in generated samples that fail to effectively reflect the multimodal nature of the data. Therefore, the shortcomings of existing stacked ensemble learning models and data generation techniques significantly restrict the prediction performance of industrial process endpoint quality indicators in small sample scenarios. Therefore, it is urgent to provide a method that can effectively improve the accuracy of endpoint quality prediction for sparse small sample data of industrial processes, overcome the limitations of traditional modeling techniques, and provide reliable technical support for industrial intelligent production decision-making. Summary of the Invention

[0004] In response to the problems existing in the above-mentioned prior art, the present invention provides a method for predicting the quality of sparse small-sample industrial processes based on a stacked network model. The method has a simple implementation process, a high degree of intelligence, and ideal prediction accuracy. It can achieve high-precision prediction of the endpoint quality indicators of complex industrial generation processes, help support the application of complex industries in efficient optimization and quality improvement, and is conducive to ensuring product quality and improving production efficiency.

[0005] To achieve the above object, the present invention provides a method for predicting the quality of sparse small samples of industrial processes based on a stacked network model, comprising the following steps: Step 1: Collect data from complex industrial production processes; Use high-precision sensors and industrial recording systems to collect quality report data during industrial production processes; Step 2: Stratify the data according to the missing rate and eliminate abnormal data by combining the interquartile range method and production experience; Step 3: Use a conditional generative adversarial network to generate synthetic data to expand the small sample dataset, construct a dataset, and then divide the dataset into a training set and a test set; Step 4: Obtain the first-level basic model based on the cumulative contribution rate screening method of SHAP value; Step 5: Build the first layer of the stacked ensemble learning model and tune the hyperparameters using the Bayesian optimization algorithm; Step 6: Use the Ridge meta-learner to integrate the output of the basic model and build the second layer network; Step 7: Use the 6-fold cross-validation strategy to train the model with the training set; Step 8: Quantify the prediction performance of the target model using multiple indicators based on the test set to obtain a quality prediction model; Step 9: Based on the actual converter industrial process, use high-precision sensors to collect production process data in real time, and input the production process data into the quality prediction model as input data. Use the quality prediction model to predict and output the endpoint phosphorus content and temperature prediction results.

[0006] Furthermore, in order to more accurately remove abnormal data points, in step 2, the specific process of processing missing data and removing outliers is as follows: S21: Stratify missing data processing by variable missing rate; S22: Combine the IQR method and production experience to eliminate abnormal data. The process is as follows: A1: The IQR method eliminates outliers, including the following process: A11: Calculate the interquartile range of the data set using formula (1) ; (1); Where, is the first quartile, is the third quartile; A12: Use formula (2) to obtain the outlier range of the data set; (2); A13: Mark data points outside the range as outliers; A2: Combine production experience and the variable range of the actual process to further confirm and eliminate abnormal data points.

[0007] Furthermore, in order to effectively ensure the consistency and reliability of the data, in S21, the process of handling missing data in layers according to the missing rate of variables is as follows: B1: For variables with a missing rate lower than 5%, calculate the variable median and fill in the missing values. The process is as follows: B11: For variables The set of non-missing values of , sorted in ascending order, is obtained ; B12: If is an odd number, the median ;like Even number, median ; B13: Use replace Missing values in ; B2: For variables with a missing rate between 5% and 20%, the KNN interpolation method is used, and the process is as follows: B21: Choose and The missing sample is most similar A complete sample, based on the Euclidean distance through formula (3) Get similarity; (3); Where, , The samples are non-missing variables The value on B22: Yes A neighbor's The weighted average of the values is calculated and the corresponding weight is obtained by formula (4) ; (4); B23: Use Fill missing values; B3: For variables with a missing rate exceeding 20%, the corresponding variables are directly eliminated.

[0008] Furthermore, in order to more effectively alleviate the overfitting problem caused by small sample data, and at the same time, to expand the dataset size and enhance data diversity with higher quality, in step three, the specific process of using a conditional generative adversarial network to generate synthetic data is as follows: S31: Preprocess the data to adapt to the CTGAN model input requirements. The process is as follows: C1: Standardize the feature set: C11: For each variable in the expanded feature set , calculate its mean and standard deviation , normalized by formula (5); (5); Where, is the standardized variable value, , , is the sample size; C12: Standardized variables Replace the original variable , forming a standardized feature set; C2: Perform one-hot encoding on discrete variables to obtain the input data set; S32: Train the CTGAN model and generate synthetic data. The process is as follows: D1: Configure and train the CTGAN model: D11: Define the loss function of CTGAN through formula (6) and optimize the generator and the discriminator ; (6); Where, For real data, is random noise, is the conditional vector, To generate data, is the output probability of the discriminator; D12: Train the CTGAN model using the input dataset and iterate until the loss converges. D2: Generate and process synthetic data: D21: Through the trained generator , input random noise and conditional vector , generate a set of synthetic samples ; D22: Denormalize the synthetic data and restore it to the original variable scale, as shown in formula (7); (7); Where, is the normalized value of the synthetic data, is the value of the synthetic variable after restoration; D23: Verify the consistency of the statistical characteristics of the synthetic data with the real data to ensure the quality of the synthetic data.

[0009] As a preferred method, in step 4, the process of obtaining the first-level basic model based on the cumulative contribution rate screening method of SHAP value is as follows: S41: Using the data set after data augmentation in step 3, train multiple heterogeneous candidate base models, including tree-type network models, linear models, deep learning network models, and distance-based network models; S42: The output of the candidate base model is used as a feature and input into the meta-learner; S43: Apply SHAP analysis to the meta-learner and calculate the SHAP value of each base model output to quantify its marginal contribution to the final prediction. The SHAP value calculation formula is shown in formula (8); (8); Where, Features SHAP value, For all feature sets, Does not contain features A subset of To use a subset The model prediction value at time ; S44: Based on the SHAP value, calculate the cumulative contribution rate of each basic model, sort them in descending order, and select the model with a cumulative contribution rate exceeding 85%; S45: Determine the first layer of basic models and form the first layer of the stacked ensemble learning model.

[0010] Furthermore, in order to effectively balance the efficiency and performance of the constructed quality prediction model, in step 4 S5, the first layer of basic models includes K-nearest neighbor, TabTransformer, random forest, TabNet and extreme tree models.

[0011] As a preferred method, in step 5, the specific process of constructing the first layer of the stacked ensemble learning model and adjusting the hyperparameters using the Bayesian optimization algorithm is as follows: S51: Define the hyperparameter search space for each base model; S52: Use Bayesian optimization to adjust hyperparameters. Using the Gaussian process as a proxy model, we perform probabilistic modeling of the objective function value in the hyperparameter space based on the existing observation data. The process is as follows: E1: Generate initial hyperparameter combinations through random sampling and evaluate model performance; E2: Construct a Gaussian process model to predict the mean and uncertainty of the objective function, and calculate the point according to formula (9) and formula (10) respectively. Mean and variance ; (9); (10); Where, is the covariance matrix based on the kernel function, is the noise variance, is the observed value; E3: Use the square exponential kernel function to capture the smooth relationship between hyperparameters. The kernel function is defined according to formula (11): ; (11); Where, is the signal variance, is the length scale, used to control the smoothness of the function; E4: Define the expected improvement acquisition function according to formula (12) , and improve formula (12) to get formula (13), and at the same time, improve the acquisition function by expectation Select the next set of hyperparameters; (12); Where, is the current optimal objective function value; (13); Where, and are the cumulative distribution function and probability density function of the standard normal distribution respectively; E5: Iteratively evaluate the model performance and update the Gaussian process posterior distribution until the convergence condition is met or the maximum number of iterations is reached; S53: Apply the optimized hyperparameters to the first layer of the basic model.

[0012] As a preferred method, in step 7, the process of training the model using six-fold cross validation is as follows: S71: Divide the dataset into training set and test set; S72: Divide the training set into 6 mutually exclusive subsets; S73: Perform 6 training runs, using 5 subsets each time to train the model, and the remaining 1 subset is used for validation; S74: Calculate the prediction results of each validation and average the performance indicators of the six validations as the basis for evaluating the generalization ability of the model. Take the average of the prediction output of the first-layer basic model on the test set as the input feature of the meta-learner.

[0013] The present invention provides a method that can effectively improve the endpoint quality prediction accuracy of small sample data of complex industrial processes. It adopts a two-layer nonlinear process prediction network of a stacked ensemble learning model to achieve high-precision prediction of endpoint quality indicators of complex industrial production processes. For the collected historical production data, a hierarchical strategy is adopted for data preprocessing. Median filling or KNN interpolation is applied according to the variable missing rate, and outliers are eliminated by combining the interquartile range (IQR) method and production expert experience to ensure data consistency and reliability. To effectively alleviate the overfitting problem caused by small sample data, a conditional generative adversarial network (CTGAN) is used to generate synthetic samples, expand the dataset size and enhance data diversity. The stacked ensemble learning model adopts a two-layer architecture. The first layer selects K-nearest neighbor (KNN), TabTransformer, random forest (RF), TabNet and extreme tree (ET) as the base models through SHAP analysis, and ensures that the cumulative SHAP value accounts for more than 85% to balance model performance and efficiency. These models are trained using six-fold cross-validation to improve their robustness and generalization ability. Among them, KNN accurately captures the fine-grained patterns of data through local proximity and is suitable for local feature modeling. TabTransformer utilizes a multi-head self-attention mechanism to dynamically learn complex nonlinear dependencies between features, enhancing the expressive power of high-cardinality features. TabNet optimizes feature selection through hierarchical decision making and sparse masking, improving modeling efficiency for tabular data. RF effectively models nonlinear relationships by integrating multiple regression trees, reducing the risk of overfitting in a single tree. ET introduces greater randomness, reducing prediction variance and enhancing robustness to noisy data. These models work synergistically to overcome the limitations of single models in capturing complex data patterns, ensuring that critical information is not lost and significantly improving generalization. A Bayesian optimization algorithm is used to precisely tune the hyperparameters of the first-layer base model, and Gaussian process modeling is used to efficiently search for the optimal parameter combination. The second layer uses Ridge regression as a meta-learner, taking the output of the first-layer base model as input features to ultimately generate endpoint predictions. Ridge regression mitigates multicollinearity through L2 regularization, reducing the risk of overfitting and integrating the strengths of each base model to further improve the model's prediction accuracy.

[0014] The present invention overcomes the limitations of traditional modeling technology and effectively solves the shortcomings of traditional modeling methods in quality prediction of small sample production processes, such as large prediction errors and poor robustness due to data scarcity, noise interference and complex nonlinear relationships. By adopting CTGAN data enhancement technology to improve sample diversity, optimizing basic model selection using the SHAP cumulative contribution rate method, and combining Bayesian optimization and Ridge meta-learner to construct a stacked ensemble learning model, this method significantly improves the prediction accuracy and generalization ability of key variables of endpoint quality. This method can accurately predict the endpoints of chemical synthesis processes, non-ferrous metal smelting, iron and steelmaking and other production processes under closed, high temperature and high pressure conditions, providing a reliable model foundation and reliable technical support for intelligent production decision-making, process control and operation monitoring, and promoting efficient optimization and quality improvement of the production process. This method has a simple implementation process, a high degree of intelligence, and ideal prediction accuracy. It can achieve high-precision prediction of the end-point quality indicators of complex industrial production processes, help support the application of complex industries in efficient optimization and quality improvement, and is conducive to ensuring product quality and improving production efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 is a flow chart of the present invention; Figure 2 is a flowchart of Bayesian optimization in the present invention; Figure 3 This is a schematic diagram of the basic structure of CTGAN in the present invention; Figure 4 This is a comparison chart of the prediction results of different machine learning methods for phosphorus content at the end of the converter steelmaking process; Figure 5 This is a comparison chart of the prediction results of different machine learning methods for the endpoint temperature in the converter steelmaking process. DETAILED DESCRIPTION

[0016] The present invention will be further described below with reference to the embodiments and accompanying drawings.

[0017] like Figures 1 to 3 As shown, the present invention provides a method for predicting the quality of a sparse small sample industrial process based on a stacked network model, comprising the following steps: Step 1: Collect data from complex industrial production processes; Use high-precision sensors and industrial recording systems to collect quality report data during industrial production processes; Step 2: Stratify the data according to the missing rate and eliminate abnormal data by combining the interquartile range method and production experience; Step 3: Use the Conditional Tabular Generative Adversarial Network (CTGAN) to generate synthetic data to expand the small sample dataset, construct a dataset, and then divide the dataset into training and test sets; Step 4: Obtain the first-level basic model based on the cumulative contribution rate screening method of SHAP (SHapley Additive exPlanations) value; Step 5: Build the first layer of the stacked ensemble learning model and tune the hyperparameters using the Bayesian optimization algorithm; Step 6: Use the Ridge meta-learner to integrate the output of the basic model and build the second layer network; Step 7: Use the 6-fold cross-validation strategy to train the model with the training set; Step 8: Quantify the prediction performance of the target model using multiple indicators based on the test set to obtain a quality prediction model; Step 9: Based on the actual converter industrial process, high-precision sensors are used to collect production process data in real time. This production process data is then fed into a quality prediction model, which then predicts and outputs the endpoint phosphorus content and temperature. Furthermore, the outputted endpoint phosphorus content and temperature prediction results are preferably transmitted to the converter industrial system, enabling a controller in the converter industrial system to control the converter industrial production process based on the endpoint phosphorus content and temperature prediction results, thereby effectively ensuring product quality and production efficiency.

[0018] In order to more accurately remove abnormal data points, in step 2, the specific process of processing missing data and removing outliers is as follows: S21: Stratify missing data processing by variable missing rate; S22: Combine the IQR method and production experience to eliminate abnormal data. The process is as follows: A1: The IQR method eliminates outliers, including the following process: A11: Calculate the interquartile range of the data set using formula (1) (Interquartile Range); (1); Where, is the first quartile, is the third quartile; A12: Use formula (2) to obtain the outlier range of the data set; (2); A13: Mark data points outside the range as outliers; A2: Based on smelting production experience and the variable range of the actual process, if the carbon content is abnormally high or the end temperature exceeds the reasonable range, further confirm and eliminate abnormal data points.

[0019] To effectively ensure the consistency and reliability of the data, in S21, the process of handling missing data in layers according to the missing rate of variables is as follows: B1: For variables with a missing rate lower than 5%, calculate the variable median and fill in the missing values. The process is as follows: B11: For variables The set of non-missing values of , sorted in ascending order, is obtained ; B12: If is an odd number, the median ;like Even number, median ; B13: Use replace Missing values in ; B2: For variables with a missing rate between 5% and 20%, the KNN interpolation method is used, and the process is as follows: B21: Choose and The missing sample is most similar A complete sample, based on the Euclidean distance through formula (3) Get similarity; (3); Where, , The samples are non-missing variables The value on B22: Yes A neighbor's The weighted average of the values is calculated and the corresponding weight is obtained by formula (4) ; (4); B23: Use Fill missing values; B3: For variables with a missing rate exceeding 20%, the corresponding variables are directly eliminated.

[0020] In order to more effectively alleviate the overfitting problem caused by small sample data, and at the same time, to expand the dataset size and enhance data diversity with higher quality, in step three, the specific process of using a conditional generative adversarial network to generate synthetic data is as follows: S31: Preprocess the data to adapt to the CTGAN model input requirements. The process is as follows: C1: Standardize the feature set: C11: For each variable in the expanded feature set , calculate its mean and standard deviation , normalized by formula (5); (5); Where, is the standardized variable value, , , is the sample size; C12: Standardized variables Replace the original variable , forming a standardized feature set; C2: Perform one-hot encoding on discrete variables to obtain the input data set; S32: Train the CTGAN model and generate synthetic data. The process is as follows: D1: Configure and train the CTGAN model: D11: Define the loss function of CTGAN through formula (6) and optimize the generator and the discriminator ; (6); Where, For real data, is random noise, is the conditional vector, To generate data, is the output probability of the discriminator; D12: Train the CTGAN model using the input dataset and iterate until the loss converges. D2: Generate and process synthetic data: D21: Through the trained generator , input random noise and conditional vector , generate a set of synthetic samples ; D22: Denormalize the synthetic data and restore it to the original variable scale, as shown in formula (7); (7); Where, is the normalized value of the synthetic data, is the value of the synthetic variable after restoration; D23: Verify the consistency of the statistical properties of the synthetic data (such as mean, variance, distribution) with the real data to ensure the quality of the synthetic data.

[0021] In order to effectively balance the efficiency and performance of the constructed quality prediction model, in step 4 S5, the first layer of basic models includes K nearest neighbor, TabTransformer, random forest, TabNet and extreme tree models.

[0022] As a preferred method, in step 4, the process of obtaining the first-level basic model based on the cumulative contribution rate screening method of SHAP value is as follows: S41: Using the data set after data augmentation in step 3, train multiple heterogeneous candidate base models, including various tree type network models, linear models, deep learning network models, and distance-based network models; S42: The output of the candidate base model is used as a feature and input into the meta-learner; S43: Apply SHAP analysis to the meta-learner and calculate the SHAP value of each base model output to quantify its marginal contribution to the final prediction. The SHAP value calculation formula is shown in formula (8); (8); Where, Features SHAP value, For all feature sets, Does not contain features A subset of To use a subset The model prediction value at time ; S44: Based on the SHAP value, calculate the cumulative contribution rate of each basic model, sort them in descending order, and select the model with a cumulative contribution rate exceeding 85%; S45: Determine the first layer of basic models and form the first layer of the stacked ensemble learning model.

[0023] As a preferred method, in step 5, the specific process of constructing the first layer of the stacked ensemble learning model and adjusting the hyperparameters using the Bayesian optimization algorithm is as follows: S51: Define the hyperparameter search space for each base model; S52: Bayesian optimization is used to adjust hyperparameters. Using the Gaussian Process (GP) as a proxy model, the objective function value in the hyperparameter space is probabilistically modeled based on existing observational data. The process is as follows: E1: Generate initial hyperparameter combinations through random sampling and evaluate model performance; E2: Construct a Gaussian process model to predict the mean and uncertainty of the objective function, and calculate the point according to formula (9) and formula (10) respectively. Mean and variance ; (9); (10); Where, is the covariance matrix based on the kernel function, is the noise variance, is the observed value; E3: Use the square exponential kernel function to capture the smooth relationship between hyperparameters. The kernel function is defined according to formula (11): ; (11); Where, is the signal variance, is the length scale, used to control the smoothness of the function; E4: Define the expected improvement acquisition function according to formula (12) , and improve formula (12) to get formula (13), and at the same time, improve the acquisition function by expectation Select the next set of hyperparameters; (12); Where, is the current optimal objective function value; (13); Where, and are the cumulative distribution function and probability density function of the standard normal distribution respectively; E5: Iteratively evaluate the model performance and update the Gaussian process posterior distribution until the convergence condition is met or the maximum number of iterations is reached; S53: Apply the optimized hyperparameters to the first layer of the basic model.

[0024] As a preferred method, in step 7, the process of training the model using six-fold cross validation is as follows: S71: Divide the dataset into training set and test set; S72: Divide the training set into 6 mutually exclusive subsets; S73: Perform 6 training runs, using 5 subsets each time to train the model, and the remaining 1 subset is used for validation; S74: Calculate the prediction results of each validation and average the performance indicators of the six validations as the basis for evaluating the generalization ability of the model. Take the average of the prediction output of the first-layer basic model on the test set as the input feature of the meta-learner.

[0025] Verification analysis: During the data preprocessing phase, high-precision sensors and industrial recording systems were used to collect data from the converter steelmaking process. This data covered chemical composition, process parameters, material usage, oxygen consumption, ore and auxiliary materials, and solid waste. The final report data was generated based on actual converter production data from a steel plant. To ensure data integrity and quality, a series of data preprocessing measures were implemented. A median-filling method was used to address small missing values for seven features: oxygen consumption, furnace solid waste pellets, iron ladle solid waste pellets, carbon content percentage in steel, manganese content percentage in steel, phosphorus content percentage in steel, and sulfur content percentage in steel. KNN interpolation was used to fill in low-missing data for three features: lime usage, magnesium pellet usage, and solid waste pellet usage. Five features with high missing rates were removed: limestone, dolomite, pellets, return ore, and lump ore. Furthermore, two features, sinter return ore and ground sinter ore, were removed because their values were all zero, lacking variability and contributing to the model. After processing, the dataset retained 29 features, including 19 features without missing values, 7 features imputed using the median, and 3 features imputed using KNN. This preprocessing significantly improved the integrity and quality of the dataset by removing variables with high missing rates and filling in missing values with appropriate methods, providing reliable input data for subsequent model training.

[0026] After missing value processing, to further improve data quality, the dataset was cleaned of outliers using a combination of the interquartile range (IQR) method and smelting experience. The IQR method calculates the interquartile range of the target variable, sets upper and lower bounds, and identifies data points outside this range as outliers. Furthermore, combined with smelting experience, data points that do not conform to actual production patterns were manually verified and removed. This process removed 127 outliers from the initial 1527 data points, ultimately retaining 1400 data points, ensuring the robustness and reliability of the dataset. To address the issue of small sample sizes, a conditional generative adversarial network (CTGAN) was used to perform data augmentation on the 1400 cleaned data points. The dataset was divided into 800 training samples and 600 test samples. CTGAN data augmentation was performed only on the training set, generating 1600 synthetic data points, bringing the training set size to 2400 data points. CTGAN captures data distribution characteristics through conditional vectors and pattern-specific normalization, generating synthetic samples with highly consistent statistical properties of the original data. This effectively mitigates the risk of overfitting in small sample sizes and provides diverse feature support for subsequent training of stacked ensemble learning models. Based on the aforementioned data preprocessing and augmentation process, a dataset consisting of 3,000 data sets was constructed, consisting of 2,400 training sets and 600 test sets, for subsequent model training and validation. The dataset includes 26 input features and two output features: phosphorus content in steel ( ) and endpoint temperature ( ). By retaining key input features and removing features with high missingness or no variability, the input data fully reflects the nonlinear relationships and dynamic characteristics of the converter steelmaking process, providing a high-quality training foundation for the model. To improve the model's generalization and prediction accuracy, six-fold cross-validation was used for training and validation of the training set data to ensure the model's stability and robustness on different data subsets. Utilizing the SHAP cumulative contribution rate technique, based on the Shapley value from game theory, the marginal contribution of candidate base models is quantified. Model combinations with cumulative SHAP values exceeding 85% are selected to optimize computational efficiency and predictive performance. The stacked ensemble learning model utilizes a two-layer architecture: the first layer comprises selected KNN, TabTransformer, RF, TabNet, and ET base models to capture the multidimensional nonlinear patterns of the data. The second layer uses Ridge regression as a meta-learner, integrating the outputs of the base models through L2 regularization to mitigate multicollinearity. Hyperparameter optimization utilizes a Bayesian optimization algorithm to efficiently search for optimal hyperparameter combinations and improve model performance.

[0027] In order to systematically evaluate the performance of the stacked ensemble learning model in the steelmaking process prediction task, five indicators, namely root mean square error (RMSE), coefficient of determination (R²), and prediction error variance (Var), are used for quantitative evaluation. The formulas are: (14); (15); (16); in, is the sample size, is the predicted value, is the actual value, These indicators evaluate the predictive ability of the model from different perspectives. This measure measures the goodness of fit of the model; values closer to 1 indicate a stronger model's ability to explain the data. RMSE measures the average deviation between the predicted and actual values. Smaller values indicate lower volatility, but are sensitive to outliers. MAE provides a more intuitive error assessment by calculating the absolute deviation and is insensitive to outliers. Smaller values indicate higher prediction accuracy. Var describes the dispersion of the prediction error; smaller values indicate greater prediction stability. By combining these three metrics, we comprehensively evaluate the model's predictive performance. Tables 1 and 2 show the comparative results of the proposed model with random forest and XGBoost. The proposed model demonstrates significant advantages in predicting the endpoint temperature of converter steelmaking and phosphorus content in steel, significantly outperforming both random forest and XGBoost. The proposed model performs best across multiple evaluation metrics: for the prediction of phosphorus content and endpoint temperature in steel, the R² reaches 0.917 and 0.929, the RMSE is 0.00094 and 2.15, and the MAE is 0.00069 and 1.47, respectively. The prediction error variance is also significantly lower than that of random forest and XGBoost. Figure 4 and Figure 5 This figure shows a comparative analysis of predicted and actual values for phosphorus content and endpoint temperature in converter steelmaking, with the sample number plotted on the horizontal axis and the phosphorus content and temperature values plotted on the vertical axis. The red line in the figure represents the predictions of the proposed model, the green line represents the Random Forest (RF) model, and the blue line represents the XGBoost model. As can be seen from the figure, the proposed model's predictions closely match the actual values, exhibiting minimal fluctuation and demonstrating the most stable prediction performance. However, the predictions of RF and XGBoost exhibit significant fluctuations, particularly at certain sample points, demonstrating the superiority of the proposed model in both prediction accuracy and stability.

[0028] This paper proposes a two-layer nonlinear process prediction network that integrates a stacked ensemble learning model, aiming to achieve high-precision prediction of endpoint quality for small, sparse data samples from industrial processes. A hierarchical data processing strategy is employed for collected historical production data. Median filling or KNN interpolation is applied based on the missing variable rate, and outliers are removed by combining the interquartile range (IQR) method and smelting expert experience to ensure data consistency and reliability. To alleviate the overfitting problem caused by small sample data, a conditional generative adversarial network (CTGAN) is used to generate synthetic samples, expanding the dataset size and enhancing data diversity. The stacked ensemble learning model employs a two-layer architecture. The first layer uses SHAP analysis to select K-nearest neighbor (KNN), TabTransformer, random forest (RF), TabNet, and extreme tree (ET) as base models, ensuring that the cumulative SHAP value exceeds 85% to balance model performance and efficiency. These models are trained using six-fold cross-validation to enhance their robustness and generalization. KNN accurately captures fine-grained patterns in the data through local proximity, making it suitable for local feature modeling. TabTransformer utilizes a multi-head self-attention mechanism to dynamically learn complex nonlinear dependencies between features, enhancing the expressive power of high-cardinality features. TabNet optimizes feature selection through hierarchical decision making and sparse masking, improving modeling efficiency for tabular data. RF effectively models nonlinear relationships by integrating multiple regression trees, reducing the risk of overfitting in a single tree. ET introduces greater randomness, reducing prediction variance and enhancing robustness to noisy data. These models work synergistically to overcome the limitations of single models in capturing complex data patterns, significantly improving model generalization. A Bayesian optimization algorithm is used to precisely tune the hyperparameters of the first-layer base model, and Gaussian process modeling is used to efficiently search for the optimal parameter combination. The second layer uses Ridge regression as a meta-learner, taking the output of the first-layer base model as input features to ultimately generate endpoint predictions. Ridge regression mitigates multicollinearity through L2 regularization, reducing the risk of overfitting and integrating the strengths of each base model to further improve prediction accuracy. This method overcomes the limitations of traditional modeling techniques, effectively addressing the large prediction errors and poor robustness of traditional modeling methods for predicting small-sample converter steel quality due to data scarcity, noise interference, and complex nonlinear relationships. By employing CTGAN data augmentation technology to increase sample diversity, optimizing basic model selection using the SHAP cumulative contribution rate method, and combining Bayesian optimization with the Ridge meta-learner to construct a stacked ensemble learning model, this method significantly improves the prediction accuracy and generalization ability of key endpoint quality variables.This method can accurately predict the end point of chemical synthesis processes, non-ferrous metal smelting, iron and steelmaking and other production processes under closed, high-temperature and high-pressure conditions, providing a reliable model foundation and reliable technical support for intelligent production decision-making, process control and operation monitoring, and supporting the metallurgical industry's application in efficient optimization and quality improvement.

Claims

1. A method for predicting the quality of sparse small sample industrial processes based on a stacked network model, characterized in that: The following steps are involved: Step 1: Collect data from complex industrial production processes; Use high-precision sensors and industrial recording systems to collect quality report data during industrial production processes; Step 2: Stratify the data according to the missing rate and eliminate abnormal data by combining the interquartile range method and production experience; Step 3: Use a conditional generative adversarial network to generate synthetic data to expand the small sample dataset, construct a dataset, and then divide the dataset into a training set and a test set; Step 4: Obtain the first-level basic model based on the cumulative contribution rate screening method of SHAP value; Step 5: Build the first layer of the stacked ensemble learning model and tune the hyperparameters using the Bayesian optimization algorithm; Step 6: Use the Ridge meta-learner to integrate the output of the basic model and build the second layer network; Step 7: Use the 6-fold cross-validation strategy to train the model with the training set; Step 8: Quantify the prediction performance of the target model using multiple indicators based on the test set to obtain a quality prediction model; Step 9: Based on the actual converter industrial process, use high-precision sensors to collect production process data in real time, and input the production process data into the quality prediction model as input data. Use the quality prediction model to predict and output the endpoint phosphorus content and temperature prediction results.

2. The method for predicting the quality of sparse small sample industrial processes based on a stacked network model according to claim 1 is characterized in that: In step 2, the specific process of handling missing data and removing outliers is as follows: S21: Stratify missing data processing by variable missing rate; S22: Combine the IQR method and production experience to eliminate abnormal data. The process is as follows: A1: The IQR method eliminates outliers, including the following process: A11: Calculate the interquartile range of the data set using formula (1) ; （1）； Where, is the first quartile, is the third quartile; A12: Use formula (2) to obtain the outlier range of the data set; （2）； A13: Mark data points outside the range as outliers; A2: Combine production experience and the variable range of the actual process to further confirm and eliminate abnormal data points.

3. The method for predicting the quality of sparse small sample industrial processes based on a stacked network model according to claim 2, characterized in that: In S21, the process of handling missing data stratified by variable missing rate is as follows: B1: For variables with a missing rate lower than 5%, calculate the variable median and fill in the missing values. The process is as follows: B11: For variables The set of non-missing values of , sorted in ascending order, is obtained ; B12: If is an odd number, the median ;like Even number, median ; B13: Use replace Missing values in ; B2: For variables with a missing rate between 5% and 20%, the KNN interpolation method is used, and the process is as follows: B21: Choose and The missing sample is most similar A complete sample, based on the Euclidean distance through formula (3) Get similarity; （3）； Where, , The samples are non-missing variables The value on B22: Yes A neighbor's The weighted average of the values is calculated and the corresponding weight is obtained by formula (4) ; （4）； B23: Use Fill missing values; B3: For variables with a missing rate exceeding 20%, the corresponding variables are directly eliminated.

4. The method for predicting the quality of sparse small sample industrial processes based on a stacked network model according to claim 1, characterized in that: In step three, the specific process of using conditional generative adversarial networks to generate synthetic data is as follows: S31: Preprocess the data to adapt to the CTGAN model input requirements. The process is as follows: C1: Standardize the feature set: C11: For each variable in the expanded feature set , calculate its mean and standard deviation , normalized by formula (5); （5）； Where, is the standardized variable value, , , is the sample size; C12: Standardized variables Replace the original variable , forming a standardized feature set; C2: Perform one-hot encoding on discrete variables to obtain the input data set; S32: Train the CTGAN model and generate synthetic data. The process is as follows: D1: Configure and train the CTGAN model: D11: Define the loss function of CTGAN through formula (6) and optimize the generator and the discriminator ; （6）； Where, For real data, is random noise, is the conditional vector, To generate data, is the output probability of the discriminator; D12: Train the CTGAN model using the input dataset and iterate until the loss converges. D2: Generate and process synthetic data: D21: Through the trained generator , input random noise and conditional vector , generate a set of synthetic samples ; D22: Denormalize the synthetic data and restore it to the original variable scale, as shown in formula (7); （7）； Where, is the normalized value of the synthetic data, is the value of the synthetic variable after restoration; D23: Verify the consistency of the statistical characteristics of the synthetic data with the real data to ensure the quality of the synthetic data.

5. The method for predicting the quality of sparse small sample industrial processes based on a stacked network model according to claim 1, characterized in that: In step 4, the process of obtaining the first-level basic model based on the cumulative contribution rate screening method of SHAP value is as follows: S41: Using the data set after data augmentation in step 3, train multiple heterogeneous candidate base models, including tree-type network models, linear models, deep learning network models, and distance-based network models; S42: The output of the candidate base model is used as a feature and input into the meta-learner; S43: Apply SHAP analysis to the meta-learner and calculate the SHAP value of each base model output to quantify its marginal contribution to the final prediction. The SHAP value calculation formula is shown in formula (8); （8）； Where, Features SHAP value, For all feature sets, Does not contain features A subset of To use a subset The model prediction value at time ; S44: Based on the SHAP value, calculate the cumulative contribution rate of each basic model, sort them in descending order, and select the model with a cumulative contribution rate exceeding 85%; S45: Determine the first layer of basic models and form the first layer of the stacked ensemble learning model.

6. The method for predicting the quality of sparse small sample industrial processes based on a stacked network model according to claim 1, characterized in that: In step 4 S5, the first layer of basic models includes K-nearest neighbor, TabTransformer, random forest, TabNet and extreme tree models.

7. The method for predicting sparse small sample industrial process quality based on a stacked network model according to claim 1, characterized in that: In step 5, the specific process of building the first layer of the stacked ensemble learning model and adjusting the hyperparameters using the Bayesian optimization algorithm is as follows: S51: Define the hyperparameter search space for each base model; S52: Use Bayesian optimization to adjust hyperparameters. Using the Gaussian process as a proxy model, we perform probabilistic modeling of the objective function value in the hyperparameter space based on the existing observation data. The process is as follows: E1: Generate initial hyperparameter combinations through random sampling and evaluate model performance; E2: Construct a Gaussian process model to predict the mean and uncertainty of the objective function, and calculate the point according to formula (9) and formula (10) respectively. Mean and variance ; （9）； （10）； Where, is the covariance matrix based on the kernel function, is the noise variance, is the observed value; E3: Use the square exponential kernel function to capture the smooth relationship between hyperparameters. The kernel function is defined according to formula (11): ; （11）； Where, is the signal variance, is the length scale, used to control the smoothness of the function; E4: Define the expected improvement acquisition function according to formula (12) , and improve formula (12) to get formula (13), and at the same time, improve the acquisition function by expectation Select the next set of hyperparameters; （12）； Where, is the current optimal objective function value; （13）； Where, and are the cumulative distribution function and probability density function of the standard normal distribution respectively; E5: Iteratively evaluate the model performance and update the Gaussian process posterior distribution until the convergence condition is met or the maximum number of iterations is reached; S53: Apply the optimized hyperparameters to the first layer of the basic model.

8. The method for predicting the quality of sparse small sample industrial processes based on a stacked network model according to claim 1, characterized in that: In step 7, the process of training the model using six-fold cross validation is as follows: S71: Divide the dataset into training set and test set; S72: Divide the training set into 6 mutually exclusive subsets; S73: Perform 6 training runs, using 5 subsets each time to train the model, and the remaining 1 subset is used for validation; S74: Calculate the prediction results of each validation and average the performance indicators of the six validations as the basis for evaluating the generalization ability of the model. Take the average of the prediction output of the first-layer basic model on the test set as the input feature of the meta-learner.

Images