Sintering mixture moisture prediction method based on multivariable interaction correlation

Abstract

The invention relates to a sintering mixture moisture prediction method based on multivariable interaction correlation, and provides an interaction correlation door mechanism-based self-correlation door and mutual door closing algorithm aiming at ignoring a coupling relation and historical self-correlation between variables in an existing time sequence prediction method. An autocorrelation door and a mutual door are introduced to display the historical correlation of mining sequences and the cross correlation between process variables. In order to solve the problem that traditional time sequence prediction only takes the accuracy of a predicted value as an evaluation index, a model loss function fusing the predicted value and trend guidance is designed to provide feedback guidance for updating of model parameters, so that model information is more suitable for time sequence prediction and downstream trend prediction tasks. In conclusion, the method provided by the invention can accurately predict the moisture and the trend of the sintered mixture, has the advantages of high credibility, high accuracy and the like, and also provides an innovative idea for time sequence prediction.

Description

Technical Field

[0001] The invention belongs to the field of sintering mixture moisture prediction, and specifically relates to a sintering mixture moisture prediction method based on multivariate interactive correlation. Background Art

[0002] Sintering is an important process to transform various powdered iron-containing raw materials into dense bodies to improve their metallurgical properties. In this process, the moisture content of the mixed materials is one of the important source parameters affecting the quality of the final product.

[0003] Appropriate moisture content can effectively improve the stability of the sintering process, ensure production continuity, reduce energy consumption, and improve the metallurgical properties of the product. However, the sintering process is complex and changeable, and various variables are intertwined, making the real-time monitoring and prediction of the moisture content of the mixture face many challenges. Therefore, it is very important to effectively predict the changes in the moisture content of the sintering mixture in the sintering project. This not only helps to improve the stability of the process, but also helps to optimize the production process and improve product quality.

[0004] At present, the more mainstream time prediction schemes in time series prediction tasks include statistical models, machine learning models, and deep learning models. Statistical models include autoregressive integrated moving average models and seasonal decomposition methods, which rely on modeling the statistical laws of historical data and are suitable for time series with clear statistical characteristics. Machine learning models, such as support vector machines and random forests, improve the model's ability to handle nonlinear relationships by introducing more complex data feature representations and learning mechanisms. In the industrial field, using deep learning models as time series predictions also plays an important role in improving production efficiency, optimizing resource allocation, and fault diagnosis and prevention.

[0005] Through a comprehensive comparative analysis of the advantages and limitations of the above-mentioned various time series prediction methods, it can be seen that the method based on deep learning has the widest scope of application and the best model effect. However, when applied to time series prediction, the current deep learning methods often ignore the coupling relationship and interaction between variables, making the accuracy of the time series prediction results low. At the same time, considering that the purpose of time series prediction is to serve subsequent trend classification tasks, such as trend classification forecasts of sintering mixture moisture and silicon content, the existing time series prediction process is often independent of subsequent downstream tasks, making the applicability and interpretability of the time series prediction results low.

[0006] Considering the sensitivity of recurrent neural networks to sequence features when processing time series data and their advantages in capturing long-term and short-term dependencies, the present invention innovatively proposes an interactive correlation gate mechanism based on dual-stage attention for time series prediction. The present invention constructs a feature representation of time series data, designs a dual-stage attention module to enhance the model's attention to key information, develops an interactive correlation gate mechanism to capture the mutual correlation of process variables and the autocorrelation of the sequence, designs a model loss function guided by time series prediction and downstream trend forecasting tasks, studies the interactive correlation mechanism of gated recurrent units, overcomes the shortcomings of traditional time series prediction models in dealing with long-term dependencies and feature correlations, and thus achieves high-precision prediction of time series.

[0007] The present application is different from the prior art as follows:

[0008] Technical comparison with patent CN201910883548.8 "A time series prediction method based on GRU neural network"

[0009] The invention proposes a time series prediction method based on GRU neural network. By using GRU-

[0010] The SES model predicts time series data, and performs secondary exponential smoothing on the obtained preliminary prediction data to obtain the final prediction data value. However, the invention fails to fully consider the complex relationship between different variables in the time series during the time series prediction process, resulting in the time series prediction results may not be able to accurately predict the time series.

[0011] Technical comparison with patent CN201910013775.5 "A traffic time series prediction method based on gated network and gradient boosting regression"

[0012] The invention proposes a method for predicting traffic time series based on gated neural network GRU and gradient boosting regression model GBR, which uses GRU to mine short-term and long-term patterns of time series data through data extraction from multiple time dimensions. However, the invention only considers the time dimension and ignores the relationship information between variables, resulting in low accuracy of time series registration and failure to predict trend information. Summary of the invention

[0013] In view of the above problems, the present invention provides a method for predicting the moisture content of sintering mixture based on multivariate interaction. The method proposed in the present invention can accurately predict the moisture content of sintering mixture and its trend, has the advantages of high credibility and high accuracy, and also provides an innovative idea for time series prediction.

[0014] To achieve the above object, the technical solution adopted by the present invention is:

[0015] A method for predicting moisture content of sintering mixture based on multivariate interaction correlation, characterized in that the specific process is as follows:

[0016] (1) Aiming at the non-stationary and long-term dependence characteristics of the sintering mixture moisture time series data, a sequence stabilization processing method is designed to simplify the difficulty of subsequent prediction tasks;

[0017] (2) Aiming at the complex relationship between process variables in the sintering industry, an embedded feature mapping method is created to provide the relationship information between multiple variables for the time series prediction model, which facilitates the construction of subsequent models;

[0018] (3) The autocorrelation gate and cross-correlation gate algorithms based on the interactive correlation gate mechanism are proposed. By improving the traditional gated recurrent unit, the autocorrelation gate and cross-correlation gate are introduced to display the historical correlation of the mining sequence itself and the cross-correlation between process variables;

[0019] (4) Design a model loss function that integrates prediction value and trend guidance to provide feedback guidance for the update of model parameters, making the model information more suitable for time series prediction and downstream trend forecasting tasks;

[0020] (5) A trend curve fitting method based on least squares is proposed to achieve accurate prediction of the mixture moisture trend classification results by combining historical and predicted data.

[0021] As a further improvement of the present invention, the step (1) is specifically as follows:

[0022] Step 1: In view of the non-stationary characteristics of the sintering mixture moisture time series data, the ADF test is first used to determine whether the moisture sequence has a unit root. If there is a unit root, the sequence is non-stationary, otherwise it is stable. Formula (1) is the model selection formula;

[0023]

[0024] Among them, y t is the original moisture sequence, Δy t is the first-order difference, d is the lag order, βt is the linear trend term, ε t is the random disturbance term;

[0025] The ADF statistic was then calculated and compared with the critical value corresponding to the 5% confidence level;

[0026] Step 2: After calculating the ADF statistic and comparing it with the critical value corresponding to the 5% confidence level, the difference between adjacent observations is calculated through the first-order difference operation, which significantly reduces the random fluctuations in the sequence, thereby enhancing the stability of the sequence and weakening the long-term dependence of the original sequence. The calculation formula is as follows:

[0027] Δyt =y t -y t-1 (2)

[0028] Among them, Δy t To represent the first-order difference sequence, y t is the sequence value at time t, y t-1 is the series value at the previous time point;

[0029] In addition, in order to verify whether the data after differential processing has weakened the long-term dependence of the original sequence, the ACF autocorrelation function is further used to analyze the correlation structure of the sequence. The calculation formula of ACF is as follows:

[0030]

[0031] Where k is the lag order, y t is the sequence value at time t, N is the sequence length, is the mean of the series.

[0032] As a further improvement of the present invention, the step (2) is specifically as follows:

[0033] Step 1: Embed the data points at the same time step into a vector. Suppose there are multivariate time series data X∈R B×T×D , B is the batch size; T is the length of the time series; D is the dimension of the variable, that is, the number of features. In order to capture the characteristic patterns at different time scales, we first select a set of block sizes {p1, p2, ..., p N}, p i represents the i-th block size and p i ≤T, and the block size p i It means that at this scale, each block contains time steps, for each block size p i , calculate the number of blocks at the current scale, the calculation formula is as follows:

[0034]

[0035] in, Indicates rounding down;

[0036] For each batch of samples X (b) ∈R T×D , b=1,2,...,B, according to the block size p i , divide it, and get

[0037] To block sequence The jth block is:

[0038]

[0039] Representation Block Contains sample X (b) From (j-1)p i +1 time step to jp i time steps of data, with dimension p i ×D, followed by each scale p i Define a linear map: Embed pi :R pi×D →R E , E is the embedding dimension, that is, the feature space dimension to which the block is mapped. For each block The mapping formula is:

[0040]

[0041] in, Represents the block matrix Expand into vectors by rows, is the scale p i The weight matrix of the embedding layer below transforms the input vector from dimension p i ×D is mapped to the embedding dimension E, b pi ∈R E is the bias vector of the embedding layer, is the embedding vector after embedding mapping; W e ∈R E×L is the weight matrix of the embedding layer, mapping the input from dimension L to the embedding dimension E; b e ∈R E is the bias vector of the embedding layer; e b,d,s ∈R E is the embedding vector after embedding mapping, representing the block The feature representation of

[0042] Through linear mapping, a multi-scale embedding vector set is obtained for all batches, all scales, and all blocks. These embedding vectors are stacked on the scale dimension to form a four-dimensional embedding tensor:

[0043]

[0044] Where N is the number of block sizes, n max =max{n i} is the maximum number of blocks at all scales;

[0045] Step 2: Then, when processing sequence data, considering the order of the sequence and the influence of position information on the model, it is necessary to add position information to the embedding vector so that the model can distinguish the features of different positions. For each segment s, a position encoding vector PE is generated.j ∈R E , whose elements are calculated according to the following formula:

[0046] i is an even number:

[0047]

[0048] i is an odd number:

[0049]

[0050] Where i∈{0,1,...,E-1} is each index of the embedding dimension;

[0051] Next, for each embedding vector The position encoding vector PE j Add it to get an embedding that includes position information:

[0052]

[0053] in, The embedded vector after adding the position encoding contains the feature representation and position information of the block;

[0054] Step 3: In order to integrate the feature representations of different scales, it is necessary to fuse the multi-scale embedding vectors. There are many options for fusion methods. Here, the average is taken along the scale dimension. For each batch b and block position j, the embedding vectors of different scales are averaged:

[0055]

[0056] in, represents the fused embedding vector at position j. By averaging the embedding vectors of different scales, the information of each scale can be integrated, so that the model can consider the features of different time scales in subsequent processing. b′=b×S+s represents the new sample index;

[0057] Combine the fused embedding vectors into a new embedding sequence to get a unified embedding tensor

[0058] Step 4: In order to capture the dependencies between different positions in the sequence, a multi-head self-attention mechanism is applied to process the fused embedded sequence. Assume that there are H attention heads in total, and the dimension of each head is d k = E / H, for the h-th attention head, define the projection matrix of query, key and value Then we have:

[0059]

[0060] in, Map it to E h The query, key, and value vector spaces of the dimension are as follows: for each attention head, the input is decomposed into three different representations Q, K, and V for calculating the attention weights;

[0061] Then, the attention weight is calculated for the h-th head. The weighted similarity between the query and the key is calculated first, and then scaled and Softmax normalized to obtain the attention matrix:

[0062]

[0063] in, is the transpose of the key matrix, The Softmax function is used to normalize the attention weights in each variable dimension. Each row of represents the degree of attention to each variable;

[0064] Apply the attention weights to the value matrix to get the output of each head:

[0065]

[0066] Concatenate the outputs of all heads along the last dimension to get the combined output matrix:

[0067]

[0068] Among them, Concat means concatenation on the last dimension;

[0069] Finally, the original embedding dimension is restored through linear mapping and residual connection is performed:

[0070]

[0071] Among them, W O is the output mapping matrix, and the residual connection helps the stability of training and the propagation of gradients.

[0072] As a further improvement of the present invention, the step (3) is specifically as follows:

[0073] Step 1: In order to capture the causal relationship between the mixture moisture series and its own historical state, the autocorrelation estimate of the mixture moisture series is calculated. Specifically, for each time step t, the autocorrelation is calculated. The formula is as follows:

[0074]

[0075] in, b p is the autocorrelation weight and bias of the mixture moisture time series, is the autocorrelation weight of the mixture moisture and the hidden state at the previous moment;

[0076] Next, in order to capture the potential temporal association between process variables and achieve more efficient joint modeling between variables, the temporal data of sintering process variables are introduced into the analysis framework. The sintering process variable matrix u is normalized to eliminate the influence of dimensional differences. The weights are learned and the dimension is reduced through convolution operations to obtain the abstract feature representation u′ of the sintering-related process variables containing more critical information. The formula is as follows:

[0077]

[0078] Among them, Conv is the convolution operation, σ is the Sigmoid activation function, T u is the number of categories of the relevant process variables, T is the sequence length, and the cross-correlation between the mixture moisture and the relevant process variable characteristics is calculated. The formula is as follows:

[0079]

[0080] in, b q are the cross-correlation weights and biases between the mixture moisture and related process variable characteristics, W h q is the cross-correlation weight between the mixture moisture and the hidden state at the previous moment;

[0081] The calculation result of the interactive correlation gate is used to update the hidden state of the model, so that the above autocorrelation and mutual correlation are learned and transmitted through the hidden state weight, so that the model can remember the global state association information. The formula is as follows:

[0082]

[0083] Among them, b h1 , b h2 The autocorrelation bias and cross-correlation bias are used for updating the hidden state of the mixture moisture, ω represents the weight hyperparameter for the autocorrelation term and the cross-correlation term, tanh[·] is the hyperbolic tangent activation function, and ⊙ is the Hadamard product operation;

[0084] Step 2: In order to capture the importance of different time steps in the time series to the prediction results, for each time step t, calculate the attention score α t , the formula is as follows:

[0085]

[0086] Among them, h tis the hidden state at the tth time step, W attn is the time attention weight vector, and then the hidden state is weighted summed to calculate the context vector c t , the formula is as follows:

[0087]

[0088] Finally, using the context vector c t The final prediction result is generated through the output layer. The specific calculation formula is as follows:

[0089] y t =W fc c t +b fc (25)

[0090] Among them, W fc is the weight matrix of the output layer, b fc is the bias vector of the output layer.

[0091] As a further improvement of the present invention, the step (4) is specifically as follows:

[0092] In order to combine the time series prediction process with downstream tasks, a model loss function that integrates the predicted value and trend guidance is designed to provide feedback guidance for the update of model parameters, making the model information more suitable for time series prediction and downstream trend forecasting tasks;

[0093] The designed model loss function L consists of two items

[0094] L=λ MSE MSE+λ trend ·L trend (26)

[0095] Among them, MSE is the mean square error loss term, which calculates the numerical error between the predicted value and the true value, L trend is the trend loss term, which measures the trend consistency between the predicted sequence and the true sequence, λ MSE and λ trend is the weight hyperparameter of the mean square error loss term and the trend loss term;

[0096] Trend loss term L trend By calculating the mean square error between the first-order difference of the predicted sequence and the true sequence, the trend consistency between the two is measured. The specific calculation is as follows:

[0097]

[0098] in, is the first-order difference of the forecast series.

[0099] As a further improvement of the present invention, the step (5) is specifically as follows:

[0100] Step 1: Determine the trend characterization window, that is, take the historical five-minute data as the observation benchmark, and combine it with the predicted value of the mixture moisture in the next three minutes. Use the sliding window method to perform local analysis on the time series data. The size of the sliding window is T, and it slides on the time series in turn. In each sliding window, the data point {(t i ,y i )} perform least squares polynomial fitting, and use constant, first-order polynomial, and second-order polynomial models for fitting:

[0101] y=a0 (28)

[0102] y=a0+a1t (29)

[0103] y=a0+a1t+a2t 2 (30)

[0104] For each model, the coefficients {a i}, so that the error between the fitted curve and the data points is minimized. Specifically, minimize the residual sum of squares:

[0105]

[0106] Where y is the actual or predicted mixture moisture data, The fitting curve is at t i The value at

[0107] Step 2: Calculate the root mean square error (RMSE) of each model and select the polynomial model with the smallest RMSE as the trend curve of the window. The calculation formula is as follows:

[0108]

[0109] Next, the selected best fitting polynomial is differentiated to obtain the first-order derivative y′ and the second-order derivative y″ for trend determination. The trend state is determined by sorting out the values ​​of the first-order derivative and the second-order derivative.

[0110] The advantages of the present invention compared with the prior art are:

[0111] (1) Aiming at the non-stationary and long-term dependence characteristics of the sintering mixture moisture time series data, the present invention proposes a sequence stabilization processing method. By performing ADF test and first-order difference processing operations on the sequence, the long-term dependence of the sintering mixture moisture sequence is weakened, the stationarity of the moisture sequence data is enhanced, and the difficulty of subsequent prediction tasks is simplified.

[0112] (2) The present invention aims at the complex relationship between process variables in the sintering industry and creates an embedded feature mapping method. By dividing the input sequence according to different scales, a multi-scale input representation is formed, which helps the model capture patterns and dependencies at different time scales, thereby improving the prediction performance of subsequent models.

[0113] (3) The present invention proposes an autocorrelation gate and a cross-correlation gate algorithm based on the interactive correlation gate mechanism. By improving the traditional gated recurrent unit, the autocorrelation gate and the cross-correlation gate are introduced to display the historical correlation of the mining sequence itself and the cross-correlation between process variables, thereby enhancing the model's ability to understand multivariate time series data.

[0114] (4) The present invention designs a model loss function that integrates prediction value and trend guidance, optimizes two target information at the same time, provides feedback guidance for the update of model parameters, and makes the model information more suitable for time series prediction and downstream trend forecasting tasks;

[0115] (5) The present invention proposes a trend curve fitting processing method based on the sliding window method and least squares, which realizes accurate classification and trend forecasting of future trends by combining historical and predicted data.

[0116] (6) The present invention uses a multi-scale feature mapping method for the first time to capture the patterns and dependencies of the sintering moisture series at different time scales, and combines it with the interactive correlation gate mechanism based on two-stage attention for optimization training, thereby achieving accurate prediction of sintering moisture. Through the sliding window method and least squares method, accurate prediction of future trends is achieved, providing a new technical path for the time series prediction of sintering moisture. BRIEF DESCRIPTION OF THE DRAWINGS

[0117] Figure 1 is a method flow chart;

[0118] Figure 2 It is a stable sintering sequence;

[0119] Figure 3 It is the experimental test effect (prediction map);

[0120] Figure 4 This is the experimental test effect (classification chart). DETAILED DESCRIPTION

[0121] The following is a detailed description of the technical solution of the application in conjunction with the accompanying drawings. The described embodiments are only part of the embodiments involved in this patent. All non-innovative embodiments of other researchers in this field based on this embodiment are within the scope of protection of this patent.

[0122] The present invention proposes a time series prediction method based on multivariate interaction correlation. Figure 1It is an implementation flow chart of the method proposed by the present invention, comprising the following steps:

[0123] (1) Construction of a stable sintering mixture moisture sequence;

[0124] Due to factors such as temperature and humidity at the sintering site and fluctuations in material ratio, the time series data of the moisture content of the mixed material usually has non-stationary characteristics. A stationary time series is a prerequisite for most time series analysis and modeling methods. A stationary time series is conducive to simplifying parameter estimation and improving prediction accuracy. In view of the non-stationary and long-term dependence characteristics of the time series data of the moisture content of the sintering mixed material, the present invention proposes a sequence stabilization processing method to simplify the difficulty of subsequent prediction tasks. Specifically, the following steps are included:

[0125] Step 1: Aiming at the non-stationary characteristics of the sintering mixture moisture time series data, the present invention first uses the ADF (Augmented Dickey-Fuller) test to determine whether the moisture sequence has a unit root. If there is a unit root, the sequence is non-stationary, otherwise it is stationary. Formula (1) is the model selection formula.

[0126]

[0127] Among them, y t is the original moisture sequence, Δy t is the first-order difference, d is the lag order, βt is the linear trend term, ε t is a random disturbance term.

[0128] Then the ADF statistic is calculated and compared with the critical value corresponding to the 5% confidence level. It can be seen from the experimental results in Table 1 that the p value is greater than the significant level, and the original hypothesis cannot be rejected. Therefore, it is determined that the mixture moisture series is non-stationary, that is, its mean and variance change over time and are not suitable for direct prediction.

[0129] Table 1 ADF test results before and after pretreatment

[0130]

[0131] Step 2: After calculating the ADF statistic and comparing it with the critical value corresponding to the 5% confidence level, the present invention calculates the difference between adjacent observations through the first-order difference operation, significantly reducing the random fluctuations in the sequence, thereby enhancing the stability of the sequence and also weakening the long-term dependence of the original sequence. The calculation formula is as follows:

[0132] Δy t =y t -y t-1 (2)

[0133] Among them, Δy t To represent the first-order difference sequence, y tis the sequence value at time t, y t-1 It is the series value at the previous time point.

[0134] In addition, in order to verify whether the data after differential processing has achieved the weakening of the long-term dependence of the original sequence, the present invention further uses ACF (autocorrelation function) to analyze the correlation structure of the sequence. The calculation formula of ACF is as follows:

[0135]

[0136] Where k is the lag order, y t is the sequence value at time t, N is the sequence length, is the mean of the series.

[0137] By comparing the autocorrelations before and after sequence preprocessing, the present invention shows that the moisture sequence before difference has a strong long-term dependence, which is manifested as a significant autocorrelation with a large lag k; while the moisture sequence after first-order difference processing has an autocorrelation coefficient that decays rapidly to zero and fluctuates within the confidence interval, indicating that the sequence has become stable. This further verifies the effectiveness of the first-order difference operation in weakening long-term dependence and enhancing stability.

[0138] In summary, a sequence stabilization processing method is proposed to simplify the difficulty of subsequent prediction tasks in view of the non-stationary and long-term dependence characteristics of sintering mixture moisture time series data.

[0139] (2) Feature mapping methods for complex variables;

[0140] Based on the stable sintering moisture sequence after the above preprocessing, the present invention aims at the complex relationship of variables in the sintering industrial process. In order to capture the multi-scale feature patterns, the present invention proposes a multi-scale embedding feature mapping method. This method introduces a multi-scale processing method before the embedding layer, divides the input sequence according to different scales, and forms a multi-scale input representation, which helps the model capture patterns and dependencies on different time scales, thereby improving the prediction performance of subsequent models. Specifically, it includes the following steps:

[0141] Step 1: Embed the data points at the same time step into a vector. Suppose there are multivariate time series data X∈R B×T×D , B is the batch size; T is the length of the time series; D is the dimension of the variable (i.e., the number of features). In order to capture the feature patterns at different time scales, the present invention first selects a set of block sizes (scales) {p1, p2, ..., p N}, p i represents the i-th block size and p i ≤T, and the block size p iIt represents the time steps contained in each block at this scale. For each block size p i , calculate the number of blocks at the current scale, the calculation formula is as follows:

[0142]

[0143] in, Indicates rounding down.

[0144] For each batch of samples X (b) ∈R T×D , b=1,2,...,B, according to the block size p i , divide it, and get

[0145] To block sequence The jth block is:

[0146]

[0147] Representation Block Contains sample X (b) From (j-1)p i +1 time step to jp i time steps of data, with dimension p i ×D. Then the present invention is for each scale p i Define a linear mapping (embedding layer): Embed pi :R pi×D →R E , E is the embedding dimension, that is, the feature space dimension to which the block is mapped. The mapping formula is:

[0148]

[0149] in, Represents the block matrix Expand into vectors by rows, is the scale p i The weight matrix of the embedding layer below transforms the input vector from dimension p i ×D is mapped to the embedding dimension E, b pi ∈R E is the bias vector of the embedding layer. is the embedding vector after embedding mapping; W e ∈R E×L is the weight matrix of the embedding layer, mapping the input from dimension L to the embedding dimension E; b e ∈R E is the bias vector of the embedding layer; e b,d,s ∈R E is the embedding vector after embedding mapping, representing the block The feature representation of .

[0150] Through linear mapping, a set of multi-scale embedding vectors is obtained for all batches, all scales, and all blocks. These embedding vectors are stacked on the scale dimension to form a four-dimensional embedding tensor:

[0151]

[0152] Where N is the number of block sizes (scales), n max =max{n i} is the maximum number of blocks at all scales.

[0153] Step 2: Then, when processing sequence data, considering the order of the sequence and the influence of position information on the model, it is necessary to add position information to the embedding vector so that the model can distinguish the features of different positions. For each segment s, a position encoding vector PE is generated. j ∈R E , whose elements are calculated according to the following formula:

[0154] i is an even number:

[0155]

[0156] i is an odd number:

[0157]

[0158] Where i∈{0,1,…,E-1} is each index of the embedding dimension.

[0159] Next, for each embedding vector The position encoding vector PE j Add it to get an embedding that includes position information:

[0160]

[0161] in, The embedded vector after adding the position encoding contains the feature representation and position information of the block.

[0162] Step 3: In order to integrate the feature representations of different scales, it is necessary to fuse the embedding vectors of multiple scales. There are many options for fusion methods. Here, we use the method of averaging along the scale dimension. For each batch b and block position j, the embedding vectors of different scales are averaged:

[0163]

[0164] in, represents the fused embedding vector at position j. By averaging the embedding vectors of different scales, the information of each scale can be integrated, so that the model can consider the features of different time scales in subsequent processing. b′=b×S+s represents the new sample index.

[0165] Combine the fused embedding vectors into a new embedding sequence to get a unified embedding tensor

[0166] Step 4: In order to capture the dependencies between different positions in the sequence, a multi-head self-attention mechanism is applied to process the fused embedded sequence. Assume that there are H attention heads, and the dimension of each head is d k =E / H, for the hth attention head, define the projection matrix of query, key and value Then we have:

[0167]

[0168]

[0169] in, Map it to E h The query, key, and value vector spaces of dimensions. This process can be understood as: for each attention head, the input is decomposed into three different representations (Q, K, V) for calculating the attention weights.

[0170] Then, the attention weight is calculated for the hth head. First, the weighted similarity between the query and the key is calculated, and then scaled and Softmax normalized to obtain the attention matrix:

[0171]

[0172] in, is the transpose of the key matrix, is a scaling factor to avoid excessive inner product values ​​in high-dimensional space, which may lead to unstable gradients. The Softmax function ensures the normalization of attention weights in each variable dimension. h (b) Each row of represents the degree of attention to each variable.

[0173] Apply the attention weights to the value matrix to get the output of each head:

[0174]

[0175] Concatenate the outputs of all heads along the last dimension to get the combined output matrix:

[0176]

[0177] Among them, Concat means concatenation on the last dimension.

[0178] Finally, the original embedding dimension is restored through linear mapping and residual connection is performed:

[0179]

[0180] Among them, W O Is the output mapping matrix. Residual connection can help the stability of training and the propagation of gradients.

[0181] In summary, the embedded feature mapping method proposed in the present invention, through multi-scale mapping embedding and multi-head attention mechanism enhancement, can provide rich and multi-scale feature representation for subsequent time series prediction and analysis models, improve the accuracy of characterization and analysis of complex relationships of multivariate time series, and thus improve the prediction performance and interpretability of industrial processes.

[0182] (3) Autocorrelation gate and cross-correlation gate algorithms that capture variable relationships;

[0183] On the basis of the multi-scale embedding feature mapping method, in order to further capture the complex relationship between different variables in the time series, this paper proposes a self-correlation gate and cross-correlation gate algorithm. The algorithm effectively models the interdependence between variables by introducing the self-correlation and cross-correlation gating mechanisms in the gated recurrent unit, and enhances the model's ability to understand multivariate time series data. The specific implementation steps are as follows:

[0184] Step 1: In order to capture the causal relationship between the mixture moisture series and its own historical state, the autocorrelation estimate of the mixture moisture series is calculated. Specifically, for each time step t, the autocorrelation is calculated The formula is as follows:

[0185]

[0186] in, b p is the autocorrelation weight and bias of the mixture moisture time series, is the autocorrelation weight between the mixture moisture and the hidden state at the previous moment.

[0187] Next, in order to capture the potential temporal associations between process variables and achieve more efficient joint modeling between variables, the temporal data of sintering process variables are introduced into the analysis framework. The sintering process variable matrix u is normalized to eliminate the impact of dimensional differences, and the weights are learned and the dimension is reduced through convolution operations to obtain the abstract feature representation u′ of the sintering-related process variables containing more critical information. The formula is as follows:

[0188]

[0189] Among them, Conv is the convolution operation, σ is the Sigmoid activation function, T u is the number of categories of the relevant process variable, and T is the sequence length. Calculate the cross-correlation between the mixture moisture and the relevant process variable characteristics The formula is as follows:

[0190]

[0191] in, b q are the cross-correlation weights and biases between the mixture moisture and related process variable characteristics, W h q is the cross-correlation weight between the mixture moisture and their respective hidden states at the previous moment.

[0192] The calculation result of the interactive correlation gate is used to update the hidden state of the model, so that the above autocorrelation and mutual correlation are learned and transmitted through the hidden state weight, so that the model can remember the global state association information. The formula is as follows:

[0193]

[0194] Among them, b h1 , b h2 are the autocorrelation bias and cross-correlation bias used for updating the hidden state of the mixture moisture, ω represents the weight hyperparameters for the autocorrelation term and the cross-correlation term, tanh[·] is the hyperbolic tangent activation function, and ⊙ is the Hadamard product operation.

[0195] Step 2: In order to capture the importance of different time steps in the time series to the prediction results, the present invention calculates the attention score α for each time step t t , the formula is as follows:

[0196]

[0197] Among them, h t is the hidden state at the tth time step, W attn is the time attention weight vector. Then the hidden state is weighted summed to calculate the context vector c t , the formula is as follows:

[0198]

[0199] Finally, using the context vector c t The final prediction result is generated through the output layer. The specific calculation formula is as follows:

[0200] y t =W fc c t +b fc (25)

[0201] Among them, W fc is the weight matrix of the output layer, b fc is the bias vector of the output layer.

[0202] In summary, the autocorrelation gate and cross-correlation gate algorithms based on the interactive correlation gate mechanism proposed in the present invention improve the traditional gated recurrent unit, introduce the autocorrelation gate and cross-correlation gate to display the historical correlation of the mining sequence itself and the cross-correlation between process variables, and effectively model the interdependence between variables.

[0203] (4) Design a model loss function that integrates time series prediction and downstream trend forecasting tasks;

[0204] In order to combine the time series prediction process with downstream tasks, the present invention designs a model loss function that integrates the predicted value and trend guidance, provides feedback guidance for the update of model parameters, and makes the model information more suitable for time series prediction and downstream trend forecasting tasks.

[0205] The model loss function L designed by the present invention consists of two items:

[0206] L=λ MSE MSE+λ trend ·L trend (26)

[0207] Among them, MSE is the mean square error loss term, which calculates the numerical error between the predicted value and the true value, L trend is the trend loss term, which measures the trend consistency between the predicted sequence and the true sequence, λ MSE and λ trend is the weight hyperparameter of the mean square error loss term and the trend loss term.

[0208] Trend loss term L trend By calculating the mean square error between the first-order difference of the predicted sequence and the true sequence, the trend consistency between the two is measured. The specific calculation is as follows:

[0209]

[0210] in, is the first-order difference of the forecast series.

[0211] In summary, the present invention designs a model loss function that integrates prediction value and trend guidance, provides feedback guidance for the update of model parameters, and enables the model to optimize two target information simultaneously during the training process, which is more suitable for time series prediction and downstream trend forecasting tasks.

[0212] (5) Forecasting mixture moisture trends based on historical and forecast data;

[0213] On the basis of the multi-step prediction of the moisture content of the sintering mixture by the above-mentioned model, in order to further analyze the trend changes of the prediction results and provide more intuitive decision support, the present invention uses the sliding window method and the least squares method based on historical data and prediction data, and polynomial fitting of the time series data of the moisture content of the mixture to classify and predict future trends. The specific implementation steps are as follows:

[0214] Step 1: Determine the trend characterization window. That is, use the historical five-minute data as the observation benchmark and combine it with the predicted value of the mixture moisture in the next three minutes. Use the sliding window method to perform local analysis on the time series data. The size of the sliding window is T, which slides on the time series in turn. In each sliding window, the data point {(t i ,y i )} perform least squares polynomial fitting, and use constant, first-order polynomial, and second-order polynomial models for fitting:

[0215] y=a0 (28)

[0216] y=a0+a1t (29)

[0217] y=a0+a1t+a2t 2 (30)

[0218] For each model, the coefficients {a i}, so that the error between the fitted curve and the data points is minimized. Specifically, minimize the residual sum of squares:

[0219]

[0220] Where y is the actual or predicted mixture moisture data, The fitting curve is at t i The value at .

[0221] Step 2: Calculate the root mean square error (RMSE) of each model and select the polynomial model with the smallest RMSE as the trend curve of the window. The calculation formula is as follows:

[0222]

[0223] Next, the selected best fitting polynomial is differentiated to obtain the first-order derivative y′ and the second-order derivative y″ for trend determination. By sorting out the values ​​of the first-order derivative and the second-order derivative, the trend state can be determined according to the rules in Table 2:

[0224] Table 2 Trend change description rules

[0225]

[0226] In summary, the present invention proposes a trend curve fitting processing method based on least squares, which combines historical and forecast data to analyze the trend changes of the forecast results and accurately classify and forecast future trends.

[0227] Example 1

[0228] This implementation is based on the sintering process of a domestic steel plant. Figure 1 First, for the moisture and variable sequences collected from the sintering site, the sequence stabilization method was used to obtain the following Figure 2 The stationary sequence shown in the figure is used to capture the patterns and dependencies at different time scales through the embedding feature mapping method, and then a sintering moisture prediction model is established based on the interactive correlation gate mechanism of two-stage attention.

[0229] In this implementation case, 330 time series of mixed material moisture content were collected at the sintering site to evaluate the model. The ratio of the training set to the test set was about 2:1. The input window and output window lengths were 5 and 3, respectively. Through rigorous experimental verification, good experimental results were achieved. Figure 3 , 4 The curve graph between the predicted value and the true value and the trend classification confusion matrix are shown respectively. In terms of the performance evaluation indicators of the model, the root mean square error is 0.0056, the mean absolute error is 0.044, the determination coefficient is 0.736, and the trend classification accuracy is 85%. The experimental results fully prove that the method proposed in the present invention performs well in sintering moisture prediction, can meet the needs of on-site work, and provides solid and reliable decision support for on-site staff.

[0230] The above description is only a preferred embodiment of the present invention and does not constitute any other form of limitation to the present invention. Any modification or equivalent change made based on the technical essence of the present invention still falls within the scope of protection required by the present invention.

Claims

1. A method for predicting moisture content of sintering mixture based on multivariate interaction, characterized in that: The specific process is as follows: (1) Aiming at the non-stationary and long-term dependence characteristics of the sintering mixture moisture time series data, a sequence stabilization processing method is designed to simplify the difficulty of subsequent prediction tasks; (2) Aiming at the complex relationship between process variables in the sintering industry, an embedded feature mapping method is created to provide the relationship information between multiple variables for the time series prediction model, which facilitates the construction of subsequent models; (3) The autocorrelation gate and cross-correlation gate algorithms based on the interactive correlation gate mechanism are proposed. By improving the traditional gated recurrent unit, the autocorrelation gate and cross-correlation gate are introduced to display the historical correlation of the mining sequence itself and the cross-correlation between process variables; (4) Design a model loss function that integrates prediction value and trend guidance to provide feedback guidance for the update of model parameters, making the model information more suitable for time series prediction and downstream trend forecasting tasks; (5) A trend curve fitting method based on least squares is proposed to achieve accurate prediction of the mixture moisture trend classification results by combining historical and predicted data.

2. The method for predicting sintering mixture moisture based on multivariate interaction according to claim 1, characterized in that: The step (1) is specifically as follows: Step 1: In view of the non-stationary characteristics of the sintering mixture moisture time series data, the ADF test is first used to determine whether the moisture sequence has a unit root. If there is a unit root, the sequence is non-stationary, otherwise it is stable. Formula (1) is the model selection formula; Among them, y t is the original moisture sequence, Δy t is the first-order difference, d is the lag order, βt is the linear trend term, ε t is the random disturbance term; The ADF statistic was then calculated and compared with the critical value corresponding to the 5% confidence level; Step 2: After calculating the ADF statistic and comparing it with the critical value corresponding to the 5% confidence level, the difference between adjacent observations is calculated through the first-order difference operation, which significantly reduces the random fluctuations in the sequence, thereby enhancing the stability of the sequence and weakening the long-term dependence of the original sequence. The calculation formula is as follows: Δy t =y t -y t-1 (2) Among them, Δy t To represent the first-order difference sequence, y t is the sequence value at time t, y t-1 is the series value at the previous time point; In addition, in order to verify whether the data after differential processing has weakened the long-term dependence of the original sequence, the ACF autocorrelation function is further used to analyze the correlation structure of the sequence. The calculation formula of ACF is as follows: Where k is the lag order, y t is the sequence value at time t, N is the sequence length, is the mean of the series.

3. The method for predicting sintering mixture moisture based on multivariate interaction according to claim 1, characterized in that: The step (2) is specifically as follows: Step 1: Embed the data points at the same time step into a vector. Suppose there are multivariate time series data X∈R B ×T×D , B is the batch size; T is the length of the time series; D is the dimension of the variable, that is, the number of features. In order to capture the characteristic patterns at different time scales, we first select a set of block sizes {p1, p2, ..., p N }, p i represents the i-th block size and p i ≤T, and the block size p i It means that at this scale, each block contains time steps, for each block size p i , calculate the number of blocks at the current scale, the calculation formula is as follows: in, Indicates rounding down; For each batch of samples X (b) ∈R T×D , b=1,2,...,B, according to the block size p i , divide it, have to To block sequence The jth block is: Representation Block Contains sample X (b) From (j-1)p i +1 time step to jp i time steps of data, with dimension p i ×D, followed by each scale p i Define a linear map: Embed pi :R pi×D →R E , E is the embedding dimension, that is, the feature space dimension to which the block is mapped. For each block The mapping formula is: in, Represents the block matrix Expand into vectors by rows, is the scale p i The weight matrix of the embedding layer below transforms the input vector from dimension p i ×D is mapped to the embedding dimension E, b pi ∈R E is the bias vector of the embedding layer, is the embedding vector after embedding mapping; W e ∈R E×L is the weight matrix of the embedding layer, mapping the input from dimension L to the embedding dimension E; b e ∈R E is the bias vector of the embedding layer; e b,d,s ∈R E is the embedding vector after embedding mapping, representing the block The feature representation of Through linear mapping, a multi-scale embedding vector set is obtained for all batches, all scales, and all blocks. These embedding vectors are stacked on the scale dimension to form a four-dimensional embedding tensor: Where N is the number of block sizes, n max =max{n i } is the maximum number of blocks at all scales; Step 2: Then, when processing sequence data, considering the order of the sequence and the influence of position information on the model, it is necessary to add position information to the embedding vector so that the model can distinguish the features of different positions. For each segment s, a position encoding vector PE is generated. j ∈R E , whose elements are calculated according to the following formula: i is an even number: i is an odd number: Where i∈{0,1,...,E-1} is each index of the embedding dimension; Next, for each embedding vector The position encoding vector PE j Add it to get an embedding that includes position information: in, The embedded vector after adding the position encoding contains the feature representation and position information of the block; Step 3: In order to integrate the feature representations of different scales, it is necessary to fuse the multi-scale embedding vectors. There are many options for fusion methods. Here, the average is taken along the scale dimension. For each batch b and block position j, the embedding vectors of different scales are averaged: in, represents the fused embedding vector at position j. By averaging the embedding vectors of different scales, the information of each scale can be integrated, so that the model can consider the features of different time scales in subsequent processing. b′=b×S+s represents the new sample index; Combine the fused embedding vectors into a new embedding sequence to get a unified embedding tensor Step 4: In order to capture the dependencies between different positions in the sequence, a multi-head self-attention mechanism is applied to process the fused embedded sequence. Assume that there are H attention heads in total, and the dimension of each head is d k = E / H, for the h-th attention head, define the projection matrix of query, key and value Then we have: in, Map it to E h The query, key, and value vector spaces of the dimension are as follows: for each attention head, the input is decomposed into three different representations Q, K, and V for calculating the attention weights; Then, the attention weight is calculated for the h-th head. The weighted similarity between the query and the key is calculated first, and then scaled and Softmax normalized to obtain the attention matrix: in, is the transpose of the key matrix, The Softmax function is used to normalize the attention weights in each variable dimension. Each row of represents the degree of attention to each variable; Apply the attention weights to the value matrix to get the output of each head: Concatenate the outputs of all heads along the last dimension to get the combined output matrix: Among them, Concat means concatenation on the last dimension; Finally, the original embedding dimension is restored through linear mapping and residual connection is performed: Among them, W O is the output mapping matrix, and the residual connection helps the stability of training and the propagation of gradients.

4. The method for predicting sintering mixture moisture based on multivariate interaction according to claim 1, characterized in that: The step (3) is specifically as follows: Step 1: In order to capture the causal relationship between the mixture moisture series and its own historical state, the autocorrelation estimate of the mixture moisture series is calculated. Specifically, for each time step t, the autocorrelation is calculated. The formula is as follows: in, b p is the autocorrelation weight and bias of the mixture moisture time series, is the autocorrelation weight of the mixture moisture and the hidden state at the previous moment; Next, in order to capture the potential temporal correlation between process variables and achieve more efficient joint modeling between variables, the temporal data of sintering process variables are introduced into the analysis framework. The sintering process variable matrix u is normalized to eliminate the influence of dimensional differences. The weights are learned and the dimension is reduced through convolution operations to obtain the abstract feature representation u′ of the sintering-related process variables containing more critical information. The formula is as follows: Among them, Conv is the convolution operation, σ is the Sigmoid activation function, T u is the number of categories of the relevant process variables, T is the sequence length, and the cross-correlation between the mixture moisture and the relevant process variable characteristics is calculated. The formula is as follows: in, b q are the cross-correlation weights and biases between mixture moisture and related process variable characteristics, is the cross-correlation weight between the mixture moisture and the hidden state at the previous moment; The calculation result of the interactive correlation gate is used to update the hidden state of the model, so that the above autocorrelation and mutual correlation are learned and transmitted through the hidden state weight, so that the model can remember the global state association information. The formula is as follows: Among them, b h1 、b h2 The autocorrelation bias and cross-correlation bias are used for updating the hidden state of the mixture moisture, ω represents the weight hyperparameter for the autocorrelation term and the cross-correlation term, tanh[·] is the hyperbolic tangent activation function, and ⊙ is the Hadamard product operation; Step 2: In order to capture the importance of different time steps in the time series to the prediction results, for each time step t, calculate the attention score α t , the formula is as follows: Among them, h t is the hidden state at the tth time step, W attn is the time attention weight vector, and then the hidden state is weighted summed to calculate the context vector c t , the formula is as follows: Finally, using the context vector c t The final prediction result is generated through the output layer. The specific calculation formula is as follows: y t =W fc c t +b fc (25) Among them, W fc is the weight matrix of the output layer, b fc is the bias vector of the output layer.

5. The method for predicting sintering mixture moisture based on multivariate interaction according to claim 1, characterized in that: The step (4) is specifically as follows: In order to combine the time series prediction process with downstream tasks, a model loss function that integrates the predicted value and trend guidance is designed to provide feedback guidance for the update of model parameters, making the model information more suitable for time series prediction and downstream trend forecasting tasks; The designed model loss function L consists of two items L=λ MSE ·MSE+λ trend ·L trend (26) Among them, MSE is the mean square error loss term, which calculates the numerical error between the predicted value and the true value, L trend is the trend loss term, which measures the trend consistency between the predicted sequence and the true sequence, λ MSE and λ trend is the weight hyperparameter of the mean square error loss term and the trend loss term; Trend loss term L trend By calculating the mean square error between the first-order difference of the predicted sequence and the true sequence, the trend consistency between the two is measured. The specific calculation is as follows: in, is the first-order difference of the forecast series.

6. The method for predicting moisture content of sintering mixture based on multivariate interaction according to claim 1, characterized in that: The step (5) is specifically as follows: Step 1: Determine the trend characterization window, that is, take the historical five-minute data as the observation benchmark, and combine it with the predicted value of the mixture moisture in the next three minutes. Use the sliding window method to perform local analysis on the time series data. The size of the sliding window is T, and it slides on the time series in turn. In each sliding window, the data point {(t i ,y i )} perform least squares polynomial fitting, and use constant, first-order polynomial, and second-order polynomial models for fitting: y=a0 (28) y=a0+a1t (29) y=a0+a1t+a2t 2 (30) For each model, the coefficients {a i }, so that the error between the fitted curve and the data points is minimized. Specifically, minimize the residual sum of squares: Where y is the actual or predicted mixture moisture data, The fitting curve is at t i The value at Step 2: Calculate the root mean square error (RMSE) of each model and select the polynomial model with the smallest RMSE as the trend curve of the window. The calculation formula is as follows: Next, the selected best fitting polynomial is differentiated to obtain the first-order derivative y′ and the second-order derivative y″ for trend determination. The trend state is determined by sorting out the values ​​of the first-order derivative and the second-order derivative.

Images