A cement clinker free calcium prediction method based on data enhancement transformer network

Abstract

The application discloses a cement clinker free calcium prediction method based on a data enhancement transformer network, which is applied to a cement clinker free calcium prediction task, adopts a method of combining DAWGAN and a transformer regression prediction network, applies the DAWGAN to generation of small sample data in a process industry, uses data generated by the DAWGAN to expand a data set for training the transformer regression prediction network, and uses labels generated by the DAWGAN as inputs of a decoder in the transformer regression prediction model on the basis of data enhancement. Cement production auxiliary variable data is used as inputs of an encoder, and hidden space vectors obtained through coding contain historical information and time sequence features from the auxiliary variables and are used for a multi-head self-attention layer in the decoder. The application simultaneously solves the problems of small data quantity and difficult accurate prediction caused by offline detection of cement clinker f-CaO content.

Description

Technical Field

[0001] This invention relates to a method for predicting free calcium in cement clinker based on a data-enhanced transformer network, which is applied to the task of predicting free calcium in cement clinker and belongs to the field of predicting free calcium in clinker of cement calcination systems. Background Technology

[0002] Cement is a crucial material for national infrastructure construction. Its quality directly impacts the performance and safety of infrastructure. As a key indicator of cement quality, the f-CaO content in cement clinker is of great significance for cement quality control and production process optimization. Excessive or insufficient f-CaO content can affect the stability of cement, leading to expansion cracks in cement products or concrete structures, reducing building quality, and even causing serious accidents. Currently, the f-CaO content of cement clinker is typically determined through manual, periodic sampling in an offline laboratory. This method involves long sampling intervals (usually one hour), resulting in delayed measurement results and making the measurement accuracy susceptible to human factors. This can lead to problems such as untimely adjustments and significant fluctuations in operating conditions, failing to meet the cement industry's urgent needs for clinker quality and output. Therefore, developing a method for predicting the f-CaO content of cement clinker is crucial for reducing the workload of manual offline measurements, providing real-time target parameters for the automatic control of the cement clinker calcination process, ensuring cement product quality, and achieving energy conservation and emission reduction.

[0003] Cement calcination is a typical process industry, such as Figure 1 As shown, the system exhibits complex nonlinear, strongly coupled, and time-delay characteristics, meaning the state of the system at a certain point in time may be influenced by the state at a previous point in time. First, each input variable has a different degree of influence on the f-CaO content of cement clinker, thus requiring comprehensive consideration. Second, the collected data on cement clinker production and f-CaO content during cement calcination lack consistency in measurement time. Third, the scarcity of data on cement clinker f-CaO content leads to low prediction accuracy, failing to meet the requirements for accurate real-time prediction. Early work largely focused on the impact of individual variables on cement clinker f-CaO, rather than the interactions between variables, making it difficult to highlight its distinct characteristics throughout the cement production process. For these reasons, modeling the characteristics of cement production and effectively extracting cement data features is crucial.

[0004] Most researchers today use Virtual Sample Generation (VSG) technology to solve the problem of small data volume in cement clinker f-CaO. Since Goodfellow first proposed Generative Adversarial Networks in 2014, it has been rapidly popularized. In this modeling framework, the generator and discriminator interact and learn from each other. The generator uses random noise to generate images, and the discriminator judges whether the image is real or generated. When the generated image is "real" enough, the discriminator cannot distinguish it from the real image. GAN networks have gained widespread attention from academia and industry due to their unique structure and training methods. However, the deployment of GAN networks is hindered by the instability of GANs during training. These problems include the following aspects: (1) Adversarial training: GAN training is carried out by the generator network and the discriminator network adversarially. This adversarial training makes the training process more complex and unstable. The dynamic balance between the generator and the discriminator is easily broken, resulting in training instability and difficulty in convergence. (2) Mode Collapse: In some cases, the generator will fall into a mode collapse state, that is, the diversity of generated samples is insufficient, and only similar or repeated samples can be generated. (3) Vanishing and exploding gradients: Due to the adversarial optimization between the generator and discriminator during GAN training, gradient propagation may be restricted, leading to vanishing or exploding gradients. This can cause the model parameters to fail to update effectively, making the training process difficult and unable to converge. (4) Hyperparameter sensitivity: The training process of GAN is very sensitive to hyperparameters (such as learning rate, batch size, network structure, etc.). Inappropriate hyperparameter selection may lead to unstable training or failure to converge.

[0005] Attention mechanisms can successfully capture long-term dependencies in data, focusing not only on interactions between variables but also on finding correlations between input-related variables and target variables, demonstrating powerful feature extraction capabilities. Although attention models were initially developed in machine translation, they have now become an important concept in neural network architectures. Attention processes are now widely used in tasks including text classification, computer vision, and natural language processing. Google's machine translation team completely abandoned network structures like RNNs and CNNs, proposing a model built solely by attention. In this model, the minimum number of sequence operations required to compute associations between locations is measured, and the number of operations required to compute associations between locations does not increase with distance. This model also features globally aware parallel computation, which reduces the difficulty of batch processing training samples due to memory constraints. Because of its outstanding modeling capabilities for long-distance associations and interactions in time series data, the Transformer has attracted much attention in time series modeling applications. However, in process industries, there are problems such as a lack of effective sample pairs and small sample sizes. Attention mechanisms and Transformer regression prediction models need to learn a large number of sample features for accurate predictions, which limits their application in process industries. Summary of the Invention

[0006] To address the aforementioned technical problems, this invention provides a method for predicting free calcium in cement clinker based on a data-augmented transformer network. Applied to the task of predicting free calcium in cement clinker, it combines DAWGAN with a Transformer regression prediction network. DAWGAN is applied to generate small samples of data in the process industry, and the data generated by DAWGAN is used to augment the dataset for training the Transformer regression prediction network. Based on data augmentation, the labels generated by DAWGAN are used as input to the decoder in the Transformer regression prediction model. Auxiliary variable data from cement production are used as input to the encoder. The latent space vector obtained after encoding contains historical information and temporal features from the auxiliary variables, which will be used by the multi-head self-attention layer in the decoder. Due to the introduction of the encoder-decoder framework in the Transformer model, this model can perform continuous time-series predictions of arbitrary input-output lengths, fully exploring the dependencies between continuous cement production data. The multi-head attention mechanism, in addition to focusing on the mutual influence between variables, also attempts to obtain the correlation between input process factors and target variables, achieving accurate prediction of key nodes and continuous time points in the production process. This solves both the problem of small data volume caused by offline detection of f-CaO content in cement clinker and the problem of difficulty in accurately predicting f-CaO content in cement clinker when applying soft measurement models.

[0007] To solve the above-mentioned technical problems, the technical solution adopted by the present invention is as follows:

[0008] A method for predicting free calcium in cement clinker based on data-enhanced transformer networks, the method comprising the following steps:

[0009] Step 1: Analyze the cement production process, selecting 12 variables related to the f-CaO content of cement clinker as auxiliary variables, and the f-CaO content of cement clinker Y as the predictor variable. Use the time series matrix of the 12 variables as the input to the regression prediction network, and then normalize the time series of each selected variable. The 12 variables are: kiln current average X1, secondary air temperature average X2, flue gas NOx average X3, decomposer outlet temperature 1 X4, decomposer outlet temperature 2 X5, kiln head negative pressure feedback X6, flue gas oxygen filtering X7, pressure feedback under the north grate of the second chamber X8, pressure feedback under the south grate of the second chamber X9, and decomposer coal feed rate X1. 10 High-temperature fan speed X 11 Feedback from Kiln Head Coal X 12 ;

[0010] Step 2: Perform time series matching on the cement clinker f-CaO content data according to the effective time periods of their corresponding input variables. Then, use the sliding window technique to balance the time scale between the auxiliary variables and the cement clinker f-CaO content. Normalize the data processed above, and use it as a transformer network A with dual attention mechanism data augmentation function. 2 Input to the G-Transformer network;

[0011] Step 3: Build a DAWGAN network, which includes a generator and a discriminator. Use Wasserstein distance as the loss function for the discriminator. Add channel attention and spatial attention to the generator and discriminator respectively. Input random Gaussian noise into the generator and input sample pairs formed by concatenating real variables with f-CaO data into the discriminator to train DAWGAN so that DAWGAN can generate accurate and effective sample data.

[0012] Step 4: Build a Transformer regression prediction model. Combine the DAWGAN data augmentation model with the Transformer regression prediction network. Mix and augment the generated data of DAWGAN with real data as the training dataset for the Transformer regression prediction network, and train the Transformer regression prediction network.

[0013] Step 5: Apply A 2 The regression prediction network trained in G-Transformer can be used to predict free calcium in cement clinker.

[0014] A further improvement of the technical solution of the present invention is that: the time series matrix of the 12 variables in step 1 is a two-dimensional data composed of the time series of the 12 variables in one hour, wherein the column represents the sampling data of one variable and the row represents the content value of the 12 variables in one hour.

[0015] A further improvement of the technical solution of the present invention is as follows: In step 3, both the generator and the discriminator are multi-layer deconvolutional neural network structures. At the same time, channel attention and spatial attention are added after the deconvolutional layer to increase the ability of the deconvolutional neural network to capture long-term dependencies. The input of the generator is random Gaussian noise, and the output is the concatenation of 12 auxiliary variables and label values ​​of cement clinker f-CaO data. The input of the discriminator has two parts, namely generated data and real data, and the output is the distance between the distributions of generated data and real data. The parameters are updated using the RMSprop algorithm.

[0016] A further improvement to the technical solution of this invention lies in the following: Step 3 specifically involves using random Gaussian noise as input to the generator, extracting features through a multi-layer deconvolutional neural network and dual attention, effectively capturing globally dependent features, and extracting features from both global and local perspectives to improve the quality of generated data; Wasserstein distance is used as the loss function, defined as follows:

[0017]

[0018] Among them, P r For the true data distribution, P g To generate the data distribution, f w For a discriminator network with parameter w, in WGAN, the discriminator approximates the Wasserstein distance. The distance L can indicate the training progress. The smaller the value, the smaller the Wasserstein distance between the real distribution and the generated distribution, and the better the generative adversarial network is trained.

[0019] The dual attention mechanism is used to enhance the expressive power and long-term dependency capture ability of deconvolutional neural networks. The formulas for channel attention and spatial attention are as follows:

[0020]

[0021]

[0022] Where F represents the feature data, and M represents the feature data. c M is the channel attention feature map. s For spatial attention feature maps, f 7×7The model employs a convolutional neural network, with AvgPool for average pooling and MaxPool for max pooling. Dual attention mechanisms—channel attention and spatial attention—are added to the generator and discriminator to adaptively readjust spatial and channel features, extracting feature information from both local and global perspectives. This enhances the convolutional network's ability to capture long-term dependencies, improves model stability, accelerates model convergence, ensures the generated data retains its original temporal characteristics, and provides a large amount of effective data for predicting the f-CaO content of cement clinker.

[0023] A further improvement of the technical solution of the present invention is that: in step 4, the data generated by DAWGAN in step 3 is mixed with real data to expand the training set of the Transformer regression prediction network, and then input into the Transformer regression prediction network for training. During this process, DAWGAN continuously generates a large amount of accurate and effective data to further expand the training set of the Transformer regression prediction network.

[0024] A further improvement to the technical solution of this invention lies in the following: Step 4 specifically involves using the labels generated by DAWGAN to expand the training set of the Transformer regression prediction model composed of a multi-head self-attention mechanism. The formula for the multi-head self-attention mechanism is as follows:

[0025]

[0026] The training set of the Transformer network consists of two parts: real data and data generated by DAWGAN. First, the data is positionally encoded so that the key feature matching between variables is a detailed temporal matching at the feature level. Then, the hybrid augmented dataset is fed into the encoder of the Transformer regression prediction model and finally output through the decoder to achieve the prediction of f-CaO content in cement clinker.

[0027] The technological advancements achieved by this invention due to the adoption of the above technical solutions are as follows:

[0028] 1. The present invention employs an improved transformer network with dual concern mechanism data augmentation function for the prediction of free calcium in cement clinker. In addition to improving the stability of the model, accelerating the model convergence speed, and ensuring that the generated data has real time series distribution characteristics, it also provides a considerable amount of reliable data for predicting the f-CaO content of cement clinker.

[0029] 2. This invention establishes an improved transformer network model with a dual-concern mechanism for data augmentation, which is used for the first time to perform multivariate, multidimensional time series prediction of f-CaO content in cement clinker during cement calcination. This strategy, in addition to focusing on the interactions between variables, also attempts to obtain the correlation between input process factors and the target variable to improve prediction accuracy. Attached Figure Description

[0030] Figure 1 This is a flow chart of the cement calcination process of this invention;

[0031] Figure 2 This invention is based on A 2 Structure diagram of G-Transformer's sample data enhancement and prediction model for f-CaO content in cement clinker;

[0032] Figure 3 This is a structural diagram of the DAWGAN network of this invention;

[0033] Figure 4 This is a structural diagram of the dual attention mechanism of the present invention;

[0034] Figure 5 This is a structural diagram of the Transformer regression prediction network of this invention;

[0035] Figure 6 This invention is based on A 2 Flowchart of G-Transformer's prediction model for the generation of f-CaO content in cement clinker;

[0036] Figure 7 This invention is based on A 2 Training loss curves for the generator and discriminator of G-Transformer;

[0037] Figure 8 This invention is based on A 2 Frequency histograms and normal distribution curves of the generated and real data from G-Transformer;

[0038] Figure 9 This invention is based on A 2 PCA and t-SNE dimensionality reduction plots of generated and real data from G-Transformer;

[0039] Figure 10 This invention is based on A 2 A graph showing the prediction results of G-Transformer. Detailed Implementation

[0040] The present invention will be further described in detail below with reference to embodiments:

[0041] This invention proposes an improved transformer network with a dual-concern mechanism for data augmentation, applied to the prediction of free calcium in cement clinker. The overall prediction task flowchart is as follows. Figure 6 As shown, a combination of DAWGAN and Transformer regression prediction networks is used. By reconstructing the input and output layers of the DAWGAN generator, high-dimensional random noise is mapped to low-dimensional data pairs. The relevant variables affecting clinker f-CaO content and clinker f-CaO data are concatenated according to their time scale relationship and used as the input layer of the discriminator, solving the problem of multi-time-scale imbalance. Utilizing the idea of ​​generative adversarial networks (GANs), the generator captures high-dimensional features embedded in the labeled sample data and generates reliable data, which is then used as the input to the encoder in the Transformer regression prediction model. Auxiliary variable data of cement production are used as the encoder input. The latent space vector obtained after encoding contains historical information and temporal features from the auxiliary variables, which will be used by the multi-head self-attention layer in the decoder. Due to the introduction of the encoder-decoder framework in the Transformer model, this model not only focuses on the mutual influence relationship between input variables but also seeks to obtain the correlation between input and target variables. Simultaneously, it emphasizes mining the long-term dependencies between continuous time series, achieving accurate prediction of key nodes and continuous time points in the production process. 2 The overall structure of G-Transformer is as follows: Figure 2 As shown.

[0042] Step 1: Analyze the cement rotary kiln process flow and select 12 variables related to the f-CaO content of cement clinker. First, arrange the selected variable data according to the time series. Second, process the data according to the input layer data format of the Transformer regression network. Then, normalize the data as input data for the Transformer regression prediction network.

[0043] In step 1, the production process of the cement rotary kiln is first analyzed. Twelve process parameters related to the f-CaO content of cement clinker are selected as auxiliary variables, with the f-CaO content of cement clinker as the predictor variable. The 12 selected input variables are: kiln current average value X1, secondary air temperature average value X2, flue gas NOx average value X3, decomposer outlet temperature 1 X4, decomposer outlet temperature 2 X5, kiln head negative pressure feedback X6, flue gas oxygen filter X7, pressure feedback under the north grate of the second chamber X8, pressure feedback under the south grate of the second chamber X9, and decomposer coal feed rate X1. 10 High-temperature fan speed X 11 Feedback from Kiln Head Coal X 12The f-CaO content Y in cement clinker is the output variable. The data from these 12 variables are used as the input layer data for the Transformer regression prediction model. The output variable is Y. Because the selected variables differ significantly, the data is normalized to improve model convergence speed and reduce feature loss.

[0044] Step 2: Build a Transformer regression prediction network. Input the normalized data from Step 1 into the regression prediction network, pre-train it, and save the model.

[0045] The structure of the Transformer regression prediction network is as follows: Figure 5 As shown, the data is first input into the encoder. Considering the nonlinear and coupled characteristics of cement data, the input embedding + position encoding is first fed into the encoder for multi-head self-attention, residual connections, and layer normalization. Then, the output enters the feedforward layer, undergoes two fully connected layers, residual connections, and layer normalization, and outputs the encoder result. This result, along with the output embedding + position encoding, is input into the decoder into a masked multi-head self-attention layer. At this point, only the output embedding + position encoding is used, undergoing masked multi-head self-attention, residual connections, and layer normalization, outputting a result. This result, along with the encoder result, enters the cross-attention layer, again undergoing residual connections and layer normalization, before entering the feedforward layer again. After two fully connected layers, residual connections, and layer normalization, the decoded output is obtained. After decoding, a linear transformation is required. The token embedding matrix mentioned above is actually a weight matrix. This matrix is ​​multiplied with the decoder output and then subjected to softmax to obtain the calculated data input fully connected layer. By using fully connected layers to link local features into global features, the network's feature extraction capability and prediction accuracy are improved, thus completing the forward training process of the regression prediction model.

[0046] Step 3: Construct the DAWGAN network, which consists of a generator and a discriminator. Random Gaussian noise is input into the generator, and sample pairs (concatenated with relevant variable data and real labels) are input into the discriminator. Wasserstein distance is used as the loss function for the discriminator, simultaneously indicating the training progress.

[0047] ①The structure of the DAWGAN model is as follows Figure 3As shown, part A is the generator, whose input is random noise. The DAWGAN generator is constructed from two deconvolutional layers, one convolutional layer, and multiple attention layers. It takes random noise as input and generates data as output. The specific process is as follows... Figure 3 As shown in the figure, Deconv2D, Dense, and BN represent convolutional layers, fully connected layers, and batch normalized layers, respectively. Channel Attention and Spatial Attention represent channel attention and spatial attention, respectively. The internal structures of channel attention and spatial attention are as follows: Figure 4 As shown, Tanh and LeakyReLU are activation functions. The generator structure uses a 3*3 convolution kernel with a stride of 2. After deconvolution, batch normalization is performed. The purpose of the batch normalization layer is to accelerate convergence, increase the stability of the generator, and effectively reduce gradient explosion and vanishing. Part B is the discriminator, which concatenates the relevant variables affecting the clinker f-CaO content and the clinker f-CaO data according to the time scale relationship and uses them as the input to the discriminator. In DAWGAN, the discriminator structure consists of two convolutional layers and multiple attention layers. The input is real data (X). 1～12 (Y) and generated data (X') 1～12 , Y'); Output the estimated distance between the real data and the generated data. For example Figure 3 The discriminator structure is shown in the diagram, where Conv2D, LeakyReLU, and Dense represent convolutional layers, activation functions, and fully connected layers, respectively. ChannelAttention and Spatial Attention represent channel attention and spatial attention, respectively. The input layer of the discriminator uses a 2*2 convolutional kernel, and the intermediate layers use 3*3 convolutional kernels with a stride of 2. The discriminator uses Wasserstein distance as the evaluation criterion for samples, so the sigmoid function is removed from the last layer of the discriminator. The input data passes through convolutional layers and dual attention to extract features. The gradient update algorithm RMSprop is used to update the parameters, and the absolute value of w is truncated to no more than a fixed parameter c. In this invention, c is set to 0.01. The discriminator is updated 5 times, and the generator is updated once.

[0048] ②The Wasserstein distance in DAWGAN is

[0049]

[0050] Among them, P r For the true data distribution, P g To generate the data distribution, f wFor a discriminator network with parameter w, in WGAN, the discriminator approximates the Wasserstein distance. The distance L can indicate the training progress. The smaller the value, the smaller the Wasserstein distance between the real distribution and the generated distribution, and the better the generative adversarial network is trained.

[0051] ③The loss function of the discriminator in DAWGAN

[0052]

[0053] ③Loss function of generator in DAWGAN

[0054]

[0055] The loss functions of the DAWGAN discriminator and generator are as follows: Figure 7 As shown. The methods for evaluating the quality of generated data are as follows: Figure 8 As shown in Figure 9, frequency histograms and normal distribution plots are used to demonstrate the degree of overlap between the generated data and the generated data distribution. PCA and t-SNE dimensionality reduction plots are used to transform the high-dimensional data into a low-dimensional space for evaluation. It can be seen that... Figure 8 In the frequency distribution plots of real data and generated data, the normal distribution plots show a high degree of overlap. The PCA and t-SNE dimensionality reduction plots also show a high degree of overlap between generated data and real data in the low-dimensional space.

[0056] Step 4: Build A 2 The G-Transformer model, its overall structure is as follows: Figure 2 As shown, DAWGAN is first trained 40,000 times to generate 1,000 sets of data, which are then mixed with the data processed in step 1 to form a training set. This training set is used to train the Transformer-based regression prediction network. As the amount of data increases, the training set of the regression prediction model also expands and the network parameters of the prediction model are continuously updated.

[0057] In step 4, by Figure 5 It can be seen that the training set data of the Transformer regression prediction network consists of two parts: real data and data generated by the generative adversarial network. This data is mixed with the data processed in step 1 to form the training set of the Transformer regression prediction network. The A network trained after mixing the generated and real data... 2 The G-Transformer prediction model has higher accuracy. The prediction results are as follows: Figure 10 As shown, the curve fit is quite good.

[0058] Step 5: Place A 2The regression prediction network trained in G-Transformer can be used to predict the f-CaO content of cement clinker.

[0059] This invention employs a Wasserstein Generative Adversarial Network (WGAN), where the generator's loss function focuses solely on the quality of generated samples, while the discriminator's loss function focuses on the differences between real and generated samples. This loss function design makes the training of the generator and discriminator more balanced and stable, exhibiting better continuity and differentiability, thus providing a more stable training process. Simultaneously, gradient pruning is performed on the weights of the discriminator network, limiting the range of gradients and avoiding gradient explosion, further enhancing training stability. A dual update strategy is adopted: during training, the discriminator network is updated first, followed by the generator network. By iteratively training the discriminator and generator, WGAN can better maintain the balance between the generator and discriminator, avoiding problems such as training instability and mode collapse. Furthermore, a dual attention mechanism is incorporated into the internal structure of WGAN, which can suppress the influence of noise on the discriminator's evaluation of the quality of generated data. Because the generator's input is random Gaussian noise, it affects both the generator's learning of real data features and the discriminator's discrimination process. This invention incorporates dual attention, enhancing feature extraction and reducing random noise interference, making the generated data more consistent with the real data distribution, increasing the number of effective samples, and solving the problem of data gaps and small effective sample sizes in process industries. Adding dual attention to the internal structure of WGAN further improves the stability of the generative model, increases convergence speed, and makes the training loss curve more interpretable. Simultaneously, it generates a large amount of effective data, increasing the data volume and comprehensively summarizing the characteristics of the overall industrial process, thereby expanding the training set for the next Transformer model and the feature space, enabling the Transformer model to learn from a large amount of data. Through the data augmentation in the first step, a large amount of effective data is generated, solving the problem of insufficient data for Transformer model learning, which leads to inaccurate predictions. This invention is the first to apply the Transformer model to the cement calcination process, learning the dependencies between different variable sequences, obtaining the correlation between input process factors and target variables, and thus making accurate predictions.Ultimately, the augmentation model and the prediction model constitute a system. Dual attention is incorporated into WGAN to reduce noise interference, enhance training stability, and improve the quality of generated data. Generated data from DAWGAN, consistent with the distribution of real data, is fed into the Transformer soft sensor model, expanding its training set and feature space. This allows the data features to fully reflect the entire cement calcination process, more specifically representing its characteristics. The Transformer soft sensor model fully explores the dependencies between continuous cement production data, achieving a positive promotion of the downstream prediction task from the upstream data augmentation task. The prediction performance of the Transformer soft sensor model verifies the accuracy and effectiveness of the generated data, thus validating the authenticity of the data generated in the upstream data augmentation task by evaluating the performance of the downstream prediction task.

[0060] The proposed network structure employs a dual attention mechanism and a fusion training method combining WGAN and Transformer regression prediction networks. Leveraging the powerful data generation capabilities of the WGAN generator, it is used to impute missing labeled sample data in process industries, addressing the issue of limited sample data for f-CaO content in cement clinker. The dual attention mechanism enhances the feature extraction capabilities of the generative adversarial network, improving the effectiveness and accuracy of the generated data. Based on data augmentation, DAWGAN-generated data and real data are mixed and used as input to the decoder in the Transformer regression prediction model. Cement production auxiliary variable data are used as input to the encoder; the encoded latent space vector contains historical information and temporal features from the auxiliary variables, which will be used by the multi-head self-attention layer in the decoder. Due to the introduction of the encoder-decoder framework, this model can perform continuous temporal predictions of arbitrary input-output lengths, fully exploring the dependencies between continuous cement production data. The multi-head attention mechanism, in addition to focusing on the mutual influence relationships between variables, also attempts to obtain the correlation between input process factors and target variables, achieving accurate prediction of key nodes and continuous time points in the production process. The Transformer regression prediction network can capture spatiotemporal features from the data more fully, thereby improving the performance of the regression prediction network and solving the problem of the difficulty in accurately predicting the f-CaO content of cement clinker when applying soft measurement models.

[0061] While data augmentation models and soft sensor models have been applied in process industries, research on combining them is limited, and there is a lack of effective training methods to optimize their performance and create a cohesive system. This invention addresses the issues of small effective sample size and missing labeled sample data in cement calcination. It employs DAWGAN to generate data sample pairs, resolving the problem of large sample sizes but small effective sample size in cement calcination. Simultaneously, it uses a Transformer soft sensor model to predict the f-CaO content of cement clinker, improving the prediction accuracy when applying the soft sensor model. In training the two models into a unified system, DAWGAN is first trained to Nash equilibrium, where the discriminator cannot distinguish between real and generated data. At this point, the generated data has the same distribution as the real data. The generated data, with a distribution identical to the real data, is then fed into the Transformer soft sensor model to expand its training set and feature space. This allows the data features to fully reflect the entire cement calcination process, more specifically representing its characteristics and more accurately predicting the f-CaO content of cement clinker. A joint training and evaluation method is proposed: the two models mutually promote each other. The DAWGAN-generated data expands the Transformer soft sensor model's training set, allowing it to fully explore the dependencies between continuous cement production data, thus positively promoting the downstream prediction task from the upstream data augmentation task. The prediction performance of the Transformer soft sensor model verifies the accuracy and effectiveness of the generated data, achieving the goal of verifying the authenticity of the generated data from the upstream data augmentation task by evaluating the performance of the downstream prediction task.

[0062] The above-described embodiments are merely descriptions of the methods proposed in this invention and are not intended to limit the scope of this invention. Any modifications and improvements made by those skilled in the art to the technical solutions of this invention without departing from the spirit of this invention should fall within the protection scope defined by the claims of this invention.

Claims

1. A method for cement clinker free lime prediction based on data augmentation transformer network, characterized in that: The method comprises the following steps: Step 1: Analyzing the cement production process, selecting 12 variables related to the content of cement clinker f-CaO as auxiliary variables, the content of cement clinker f-CaO Y as the predicted variable, taking the time series matrix of the 12 variables as the input of the regression prediction network, and then normalizing the time series of each selected variable; wherein the 12 variables are the average value of kiln current X1, the average value of secondary air temperature X2, the average value of smoke chamber NOX X3, the outlet temperature 1 of the decomposing furnace X4, the outlet temperature 2 of the decomposing furnace X5, the kiln head negative pressure feedback X6, the smoke chamber oxygen filter X7, the two-chamber north grate lower pressure feedback X8, the two-chamber south grate lower pressure feedback X9, the decomposing furnace coal feeding amount X 10 , the high-temperature fan speed X 11 , and the kiln head coal feedback X 12 ; Step 2: The cement clinker f-CaO content data is time series matched according to its effective time period of the corresponding input variable, and then the sliding window technique is used to balance the time scale between the auxiliary variables and the cement clinker f-CaO content. The data processed according to the above is normalized as the transformer network A with double attention mechanism data enhancement function 2 Input of G-Transformer network Step 3: build a DAWGAN network, which comprises a generator and a discriminator, adopts a Wasserstein distance as a loss function of the discriminator, adds a channel attention and a spatial attention into the generator and the discriminator respectively, inputs a random Gaussian noise into the generator, inputs a sample composed by splicing a real variable and f-CaO data into the discriminator, trains the DAWGAN, and enables the DAWGAN to generate accurate and effective sample data; Step 4: build a Transformer regression prediction model, combines the DAWGAN data enhancement model with the Transformer regression prediction network, mixes and expands the generated data of the DAWGAN and the real data as a training data set of the Transformer regression prediction network, and trains the Transformer regression prediction network; The time sequence matrix of the 12 variables in step 1 is two-dimensional data composed of the time sequences of the 12 variables in one hour, wherein a column represents sampling data of a variable, and a row represents content values of the 12 variables in one hour. Step 5: Application A 2 The regression prediction network trained in G-Transformer is used to predict the free lime in cement clinker.

2. The cement clinker free calcium prediction method based on data augmentation transformer network according to claim 1, characterized in that: In step 3, the generator and the discriminator are both multi-layer deconvolutional neural network structures, and the channel attention and the spatial attention are added after the deconvolutional layers to increase the long-term dependence capturing ability of the deconvolutional neural network; 3. The cement clinker free calcium prediction method based on data augmentation transformer network according to claim 1, characterized in that: The input of the generator is a random Gaussian noise, and the output is a splice of 12 auxiliary variables and label value cement clinker f-CaO data; the input of the discriminator has two parts, which are generated data and real data respectively, and the output is the distance between the generated data and the real data distribution; and the RMSprop algorithm is adopted to update parameters. In step 3, the specific operation is that the random Gaussian noise is taken as the input of the generator, feature extraction is performed through the multi-layer deconvolutional neural network and the double attention, global dependence features are effectively captured, feature extraction is performed from the global and local aspects, and the quality of the generated data is improved; the Wasserstein distance is taken as the loss function, and the Wasserstein distance is defined as follows:

4. The cement clinker free calcium prediction method based on data augmentation transformer network according to claim 3, characterized in that: The double attention mechanism is used to improve the expression ability and the long-term dependence capturing ability of the deconvolutional neural network, and the channel attention and the spatial attention formulas are as follows: where P r is the real data distribution, P g is the generated data distribution, f w is a discriminator network with parameters w, in WGAN, the discriminator is to approximate the Wasserstein distance, the distance L can indicate the training process, the smaller the value, the smaller the Wasserstein distance between the real distribution and the generated distribution, and the better the generative adversarial network training; In step 4, the data generated by the DAWGAN in step 3 is mixed and expanded with the real data to expand the training set of the Transformer regression prediction network, and is input into the Transformer regression prediction network for training; in this process, the DAWGAN continuously generates a large amount of accurate and effective data to continue expanding the training set of the Transformer regression prediction network. wherein F is feature data, M c is a channel attention feature map, M s is a spatial attention feature map, f 7×7 is a convolutional neural network, AvgPool is an average pooling, and MaxPool is a maximum pooling; a double attention, i.e., a channel attention and a spatial attention, is added in the generator and the discriminator; the spatial and channel features are adaptively adjusted; feature information is extracted from both local and global aspects; the ability of the convolutional network to capture long-term dependencies is increased; the stability of the model is improved; the convergence speed of the model is accelerated; the generated data has the original time sequence characteristics; and a large amount of effective data is provided for the prediction of cement clinker f-CaO content.

5. The cement clinker free calcium prediction method based on data augmentation transformer network according to claim 1, characterized in that: In step 4, the specific operation is that the label generated by the DAWGAN is used to expand the training set of the Transformer regression prediction model composed of the multi-head self-attention mechanism, and the formula of the multi-head self-attention mechanism is as follows:

6. The method of claim 5, wherein the method is based on a data augmentation transformer network. ​ The training set data of the Transformer network is composed of real data and data generated by the DAWGAN. First, the data is positionally encoded to match the key features between variables, which is a detailed time sequence matching at the feature level. Then, the mixed and expanded data set is fed into the encoder of the Transformer regression prediction model. Finally, the output is obtained through the decoder to realize the prediction of the cement clinker f-CaO content.

Images