Coke quality prediction model based on converter for samples with missing coal parameters

Abstract

The present invention is a transformer-based coke quality prediction model for samples containing missing coal parameters. First, a sample dataset containing missing values ​​of coal sample parameters is constructed. While maintaining the original missing structural information, the non-missing parameters are mapped into fixed-dimensional embedding vectors through a trainable linear transformation. Missing parameters are identified using a missing mask and replaced with a trainable placeholder embedding vector. The embedded feature sequence is input into a multi-layer stacked transformer encoder, and the contextual relationship between features is extracted using a multi-head attention mechanism and a feedforward neural network. Subsequently, a uniform fixed-length coal sample representation vector is obtained through mean pooling, and a multi-layer perceptron model simultaneously outputs the coke reactivity index (CRI) and the post-reaction strength index (CSR). The present invention can predict the hot quality of coke for sample data containing missing coal parameter values, significantly improving the applicability of the model in actual production.

Description

Technical Field

[0001] The present invention relates to the technical fields of coking, artificial intelligence and data mining processing, and in particular to a converter-based coke quality prediction model for samples containing missing coal parameters. Background Art

[0002] Over the past few decades, numerous coke quality prediction models have been proposed. From traditional statistical regression algorithms to the currently popular artificial neural networks, these methods have made significant progress in improving model prediction performance. However, existing models still face a key challenge in practical applications: when the parameters of the newly input coal sample data have missing values ​​and cannot match the complete input parameters required by the training model, the model cannot make predictions. Due to the differences in the coal sample property testing procedures adopted by different coking plants, and the different complexities and time costs of various analytical methods, the integrity of the coal sample parameters varies, and the presence of a large number of missing values ​​makes it difficult for existing models to make predictions. This limitation not only seriously restricts the application effect of the model in actual production, but also hinders the development of a global unified coke quality prediction model.

[0003] Traditional coke quality prediction models based on artificial neural networks can effectively extract the complex nonlinear relationship between coal parameters and coke quality. However, due to the fixed number of neurons in the input layer, all input coal sample data must have the same number of features and cannot contain any missing values. Otherwise, matrix operations and gradient propagation will be impossible. Therefore, how to retain the powerful feature extraction capabilities of neural networks while enabling them to handle coal parameter samples with any number of missing values, thereby improving the model's applicability and accurately and stably predicting coke quality, is a core issue in the current intelligent coal blending technology. Summary of the Invention

[0004] The purpose of the present invention is to overcome the shortcomings of the existing technology. In view of the problem that the current coke quality prediction model cannot process samples with missing coal parameters due to fixed input features, a converter-based coke quality prediction model for samples with missing coal parameters is provided.

[0005] In order to achieve the above technical objectives and the above technical effects, the present invention is implemented through the following technical solutions:

[0006] A converter-based coke quality prediction model for samples with missing coal parameters includes the following steps:

[0007] Step S1: Construction of a sample data set containing missing values ​​of coal parameters;

[0008] Step S2: embed and map the non-missing coal sample parameters, and construct missing masks and missing placeholder embeddings to represent missing features;

[0009] Step S3: Transformer encoder is constructed to extract high-dimensional features;

[0010] Step S4: Perform mean pooling processing on the extracted features and output the predicted coke quality through a multi-layer perceptron.

[0011] Furthermore, in step S1, the sample data set includes multi-dimensional coal parameters with missing values ​​and corresponding coke quality values, wherein the coke quality values ​​include reactivity CRI and post-reaction strength CSR.

[0012] Furthermore, in step S2, the missing mask length is equal to the characteristic dimension of each coal sample, which is used to identify whether each parameter is missing, wherein a mask value of 0 indicates that the parameter is missing, and a mask value of 1 indicates that the parameter exists; for non-missing coal sample parameters, the original values ​​are input into a set of trainable linear transformation layers to perform a projection operation from the scalar value to the fixed-dimensional embedding space, wherein the linear transformation layer includes a fully connected network structure consisting of a weight matrix and a bias vector, which is used to map the one-dimensional input scalar into a d-dimensional embedding vector, wherein d is a preset embedding dimension, and the weight and bias are learnable parameters, which are automatically optimized by the back-propagation algorithm during the model training process.

[0013] Furthermore, in step S2, the missing placeholder embedding is replaced by a trainable placeholder embedding vector E_miss of uniform dimension, which is automatically learned through error back propagation during model training and is used to uniformly represent the embedding form of missing features in the sample, thereby retaining the missing structural information without introducing artificial filling values.

[0014] Furthermore, in step S3, the transformer encoder includes multiple stacked encoder layers, each layer including a multi-head self-attention mechanism, a feedforward neural network module, a residual connection structure and a layer normalization module, which are used to capture high-order interaction relationships and contextual semantics between coal sample features.

[0015] Furthermore, the multi-head self-attention mechanism models the degree of influence between different coal sample parameters in the sequence by mapping the input embedding representation into query Q, key K and value V vectors respectively and calculating the attention distribution.

[0016] Furthermore, the feedforward neural network module is composed of multiple hidden layers and a single output layer, and the hidden layer is followed by a nonlinear activation function for performing position-by-position feature expansion and nonlinear modeling on each embedded vector.

[0017] Furthermore, the residual connection structure is used to perform element-wise addition operations on the input and output of each sub-module including the attention mechanism module and the feedforward neural network module, thereby effectively alleviating the problems of gradient vanishing and feature degradation in the process of deep network stacking, improving the stability and convergence performance of the model, and introducing a layer normalization mechanism based on this output to normalize each sample in its feature dimension so that its mean is zero and its variance is one, which helps to maintain the stability of the feature distribution and enhance the expressive power and training efficiency of the deep network.

[0018] Furthermore, in step S4, the mean pooling process is to average the variable-length embedded sequence features output by the transformer encoder by dimension to obtain a fixed-length coal sample representation vector of uniform dimension.

[0019] Furthermore, the multilayer perceptron consists of an input layer, one or more hidden layers and an output layer. The number of neurons in the output layer is two, one of which outputs CRI and the other outputs CSR.

[0020] The beneficial effects of the present invention are:

[0021] 1) The proposed method, which utilizes missingness masks and a trainable placeholder embedding vector mechanism, preserves missing structural information in the data without explicit interpolation or sample culling. It converts coal sample parameters with different missingness patterns into a structurally consistent vector sequence, which is then fed into a transformer encoder for medium- and high-dimensional feature extraction. Coke quality is then predicted using a multilayer perceptron. This model not only effectively addresses the issue of missing coal sample parameters and enables coke quality prediction for incomplete samples, significantly improving the model's applicability in practical applications, but also fully utilizes the effective information in the missing samples, further improving overall prediction performance.

[0022] 2) The proposed transformer encoder model, composed of a stacked multi-head self-attention mechanism and a feedforward neural network module, is capable of capturing high-order interactions and contextual dependencies between coal sample features. Compared to traditional independent feature processing methods, this transformer architecture significantly improves the ability to model nonlinear and non-stationary data, thereby enhancing the predictive performance and generalization capabilities of the coke quality prediction model. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1 A block diagram of the prediction model of the present invention;

[0024] Figure 2 is a flow chart of the prediction model of the present invention;

[0025] Figure 3 Schematic diagram of a transformer encoder model in a prediction model of the present invention;

[0026] Figure 4 Schematic diagram of a multi-layer perceptron model in the prediction model of the present invention;

[0027] Figure 5 Schematic diagram showing the comparison between the predicted values ​​and the actual values ​​of the coke CRI and CSR indicators of the prediction model of the present invention. DETAILED DESCRIPTION

[0028] The present invention will be described in detail below with reference to the accompanying drawings and in combination with embodiments.

[0029] like Figures 1 to 4A transformer-based coke quality prediction model for samples with missing coal parameters is shown. The model includes the following steps: constructing a sample dataset containing missing coal parameter values; performing embedding mapping on non-missing coal sample parameters, and constructing a missing mask and missing placeholder embedding to represent missing features; building a transformer encoder to extract high-dimensional features; performing mean pooling on the extracted features, and outputting predicted coke quality results via a multi-layer perceptron. First, embedding mapping is performed on coal sample parameters with missing values ​​and an inconsistent number of parameters in the input blended coal sample data. To this end, a missing mask vector of equal length to each coal sample feature dimension is constructed to identify whether each parameter is missing. A mask value of 0 indicates a missing parameter, and a mask value of 1 indicates a present parameter. For non-missing coal sample parameters, the original values ​​are input into a set of trainable linear transformation layers, which perform a projection operation from the scalar values ​​to a fixed-dimensional embedding space. The linear transformation layers consist of a weight matrix and a bias vector, forming a fully connected network structure that maps a one-dimensional input scalar into a d-dimensional embedding vector, where d is the preset embedding dimension. These weights and biases are learnable parameters that can be automatically optimized through error backpropagation during model training. Missing coal sample parameters are replaced with a trainable placeholder embedding vector E_miss of uniform dimension. E_miss, also a model parameter, can be learned through backpropagation during training. It is used to uniformly represent the embedding form of missing features in the sample, thereby preserving missing structural information without introducing artificial filler values. The above embedding vector sequence is then input into multiple trained stacked encoder layers. The encoder layer is a transformer encoder structure consisting of a multi-head self-attention mechanism, a feedforward neural network module, a residual connection structure, and a layer normalization module. It is used to capture high-order interactions and contextual semantics between coal sample features. The multi-head self-attention mechanism models the degree of influence between different coal sample parameters in the sequence by mapping the embedding representation into query Q, key K, and value V vectors, respectively, and calculating the attention distribution. The feedforward neural network module consists of multiple hidden layers and a single output layer, with the hidden layers followed by a nonlinear activation function (such as ReLU or GELU). The residual connection structure is used to perform element-wise addition of the input and output of each submodule (including the attention mechanism module and the feedforward neural network module). Based on this output, a layer normalization mechanism is introduced to normalize each sample across its feature dimensions to zero mean and unity. The high-dimensional embedded feature sequence extracted by the transformer encoder undergoes mean pooling, averaging the variable-length embedded sequence features output by the transformer encoder across dimensions to obtain a fixed-length coal sample representation vector of uniform dimensionality. Finally, this fixed-length representation vector is input into a trained multilayer perceptron consisting of an input layer, one or more hidden layers, and an output layer to output the predicted coke CRI and CSR indicators.

[0030] Example 1:

[0031] A dataset of 195 industrial production data was collected and the numerical representation of missing values ​​for various coal parameters (moisture, ash, volatile matter, sulfur, G value, X value, and Y value) was embedded and mapped. To this end, a missing mask vector, equal in length to each coal sample feature dimension, was constructed to identify missing parameters. A mask value of 0 indicates missing parameters, while a mask value of 1 indicates present parameters. For non-missing coal sample parameters, the original values ​​were input into a set of trainable linear transformation layers, which projected the scalar values ​​into a fixed-dimensional embedding space. The linear transformation layers consist of a weight matrix and a bias vector, forming a fully connected network structure that maps the one-dimensional input scalar into a 128-dimensional embedding vector. These weights and biases are learnable parameters that are automatically optimized during model training via error backpropagation. Missing coal sample parameters are replaced with a uniformly dimensional trainable placeholder embedding vector E_miss. E_miss is also a model parameter, learned through backpropagation during training. It is used to unify the embedding representation of missing features in the sample, thereby preserving missing structural information without introducing artificial filler values. The above embedding vector sequence is then input into three trained stacked encoder layers. The encoder layer is a transformer-encoder structure consisting of a multi-head self-attention mechanism, a feedforward neural network module, a residual connection structure, and a layer normalization module. This is used to capture high-order interactions and contextual semantics between coal sample features. The multi-head self-attention mechanism models the influence between different coal sample parameters in the sequence by mapping the embedding representation into query Q, key K, and value V vectors and calculating the attention distribution. The feedforward neural network module consists of three hidden layers and a single output layer, followed by a nonlinear activation function (ReLU). The residual connection structure performs element-wise addition between the input and output of each submodule (including the attention mechanism module and the feedforward neural network module). Layer normalization is then applied to these outputs to normalize each sample across its feature dimensions to have a mean of zero and a variance of one. The high-dimensional embedded feature sequence extracted by the transformer encoder is subjected to mean pooling. This involves averaging the variable-length embedded sequence features output by the transformer encoder by dimension to obtain a fixed-length coal sample representation vector of uniform dimension. Finally, this fixed-length representation vector is input into a trained multilayer perceptron consisting of one input layer, three hidden layers, and one output layer to output the predicted coke CRI and CSR indicators. Figure 5The thermal quality parameters of coke predicted by the model of the present invention and the thermal quality parameters of coke actually produced by industrial coke ovens. As can be seen from the figure, the prediction model built using the present invention can successfully predict sample data containing missing values ​​of coal parameters. The average absolute error value of the predicted coke CRI and CSR indicators and the thermal quality parameters of industrially produced coke is 3.88, and the prediction results are very close to the true values.

[0032] Comparative Example 1:

[0033] The 195 industrial production data sets used in Example 1 were selected and numerically represented with varying coal parameters (moisture, ash, volatile matter, sulfur, G value, X value, and Y value) and missing values ​​were fed into an artificial neural network model consisting of three fully connected layers. The last fully connected layer contained two neurons, one for CRI and one for CSR. Because the input layer of the artificial neural network model has a fixed number of neurons, all input coal sample data must have the same number of features and must not contain any missing values. Therefore, it is impossible to predict the coke CRI and CSR indicators based on this industrial production data set.

[0034] In summary, the method provided by the present invention for embedding and mapping the numerical parameters in the coal sample and combining the missing information modeling mechanism with the converter encoder structure for feature extraction can realize the prediction of the coke thermal quality of sample data containing missing values ​​of coal parameters, significantly improving the applicability of the model in actual production, and also promoting the development of a global unified coke quality prediction model.

[0035] The foregoing description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Those skilled in the art will readily appreciate that various modifications and variations of the present invention are possible. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of the present invention are intended to be within the scope of protection of the present invention.

Claims

1. A converter-based coke quality prediction model for samples with missing coal parameters, characterized in that: The model consists of the following steps: Step S1: Construction of a sample data set containing missing values ​​of coal parameters; Step S2: embed and map the non-missing coal sample parameters, and construct a missing mask and a missing placeholder embedding to represent the missing features; wherein the missing mask length is equal to the feature dimension of each coal sample, and is used to identify whether each parameter is missing, wherein a mask value of 0 indicates that the parameter is missing, and a mask value of 1 indicates that the parameter exists; for the non-missing coal sample parameters, a projection operation from the scalar value to the fixed-dimensional embedding space is performed by inputting the original value into a set of trainable linear transformation layers, wherein the linear transformation layer includes a fully connected network structure consisting of a weight matrix and a bias vector, which is used to map the one-dimensional input scalar into a d-dimensional embedding vector, wherein d is a preset embedding dimension, and the weight and bias are learnable parameters, which are automatically optimized by the back-propagation algorithm during the model training process; Step S3: construct a transformer encoder in which each layer is composed of a multi-head self-attention mechanism, a feedforward neural network module, a residual connection structure and a layer normalization module, which is used to extract high-dimensional features and capture high-order interactive relationships and contextual semantics between coal sample features; wherein the residual connection structure is used to perform element-wise addition operations on the input and output of each submodule, thereby effectively alleviating the problems of gradient vanishing and feature degradation in the process of deep network stacking, improving the stability and convergence performance of the model, and introducing a layer normalization mechanism based on this output to normalize each sample in its feature dimension so that its mean is zero and its variance is one, so as to maintain the stability of the feature distribution and enhance the expression ability and training efficiency of the deep network; Step S4: Perform mean pooling processing on the extracted features and output the predicted coke quality through a multi-layer perceptron.

2. The converter-based coke quality prediction model for samples with missing coal parameters according to claim 1, characterized in that: In step S1, the sample data set includes multi-dimensional coal parameters with missing values ​​and corresponding coke quality values, wherein the coke quality values ​​include reactivity CRI and post-reaction strength CSR.

3. The converter-based coke quality prediction model for samples with missing coal parameters according to claim 1, characterized in that: In step S2, the missing placeholder embedding is replaced by a trainable placeholder embedding vector E_miss of uniform dimension. E_miss is automatically learned through error back propagation during model training and is used to uniformly represent the embedding form of missing features in the sample, thereby retaining the missing structural information without introducing artificial filling values.

4. The converter-based coke quality prediction model for samples with missing coal parameters according to claim 1, characterized in that: The multi-head self-attention mechanism models the degree of influence between different coal sample parameters in the sequence by mapping the input embedding representation into query Q, key K and value V vectors respectively and calculating the attention distribution.

5. The converter-based coke quality prediction model for samples with missing coal parameters according to claim 1, characterized in that: The feedforward neural network module consists of multiple hidden layers and a single output layer. The hidden layer is followed by a nonlinear activation function, which is used to perform position-by-position feature expansion and nonlinear modeling on each embedded vector.

6. The converter-based coke quality prediction model for samples with missing coal parameters according to claim 1, characterized in that: In step S4, the mean pooling process is to average the variable-length embedded sequence features output by the transformer encoder according to the dimensions to obtain a fixed-length coal sample representation vector of uniform dimension.

7. The converter-based coke quality prediction model for samples with missing coal parameters according to claim 6, characterized in that: The multilayer perceptron consists of an input layer, one or more hidden layers and an output layer. The number of neurons in the output layer is two, one of which outputs CRI and the other outputs CSR.

Images