Prediction method of key parameters of industrial processes based on spatiotemporal graph convolutional network

Abstract

This invention addresses the time lag, data redundancy, and missing information problems inherent in industrial process data by providing a method for predicting key parameters of industrial processes based on a spatiotemporal graph convolutional network. This method first preprocesses and sequence-partitions the collected industrial process data. It then models the internal features of subsequences in the variable dimension through global and local graph convolutions. Finally, it completes temporal modeling between subsequences in the time dimension through continuous-time modeling and multi-head attention. The continuous-time modeling consists of multiple neural ordinary differential equations and a gating mechanism, which loops through the modeling of each subsequence across the entire time dimension. Finally, it uses a multilayer perceptron to predict key parameters of the industrial process. This invention addresses the existing problems in industrial process data modeling, improves the prediction accuracy of key parameter data for industrial processes, and provides technical support for the optimal control of industrial processes.

Description

Technical Field

[0001] The present invention belongs to the field of industrial time series prediction, and specifically is a method for predicting key parameters of industrial processes based on a spatiotemporal graph convolutional network. Background Art

[0002] In modern industry, data-driven models offer broad application prospects for improving production efficiency and optimizing resource utilization. Deep learning and machine learning have become essential tools for processing industrial data. In recent years, advancements in data acquisition technology have significantly increased the amount of industrial process data available, highlighting the limitations of shallow neural networks in capturing data dependencies. Consequently, deep learning methods have become crucial for achieving accurate predictions. However, despite significant performance improvements, existing deep learning models still face challenges in addressing the unique characteristics and complexity of industrial data, which often hinder their stability and generalization capabilities in practical applications.

[0003] First, in industrial scenarios, there are often complex nonlinear relationships between variables. For example, physical quantities such as temperature, pressure, and flow affect each other. In recent years, graph neural networks have made progress in extracting potential dependencies, effectively solving this problem. For example, the adaptive graph convolutional recurrent network (AGCRN) captures process-related features through a learnable adjacency matrix, improving prediction accuracy and interpretability; the meta-graph learning network (MegaCRN) combines graph networks with meta-learning to enhance the adaptability and generalization ability of the model in complex scenarios. However, in actual industrial data, there is often a lag effect between variables. For example, in the tobacco rehumidification process, there is a time delay between the outlet moisture and other variables, and existing methods mostly assume that the variable relationship is synchronous, ignoring this delayed transmission effect, resulting in a decrease in prediction accuracy.

[0004] Secondly, due to the complexity of sensor settings and working conditions, industrial process data often has significant redundancy. Some data need to reach a threshold to affect quality parameters, while others are only relevant to specific variables. The attention mechanism filters irrelevant information through weight distribution, effectively alleviating the problem of data redundancy. For example, the block temporal attention network (PatchTST) and the interactive attention network (iTransformer) capture key interactions through dot product attention and extract target-related features; the fully connected spatiotemporal graph neural network (FCSTGNN) introduces an attention mechanism into the graph network to adaptively highlight important neighbor information. However, industrial data has high noise due to sensor errors, environmental influences, etc., and the attention mechanism is sensitive to noise, which may lead to unstable model predictions.

[0005] Finally, industrial data, influenced by complex environments, often exhibits discreteness, sparsity, and a lack of continuity. Sensors are susceptible to damage under conditions of high temperature, vibration, or corrosion, leading to interrupted data collection. Sparsity is further exacerbated by placement restrictions and the lack of monitoring at key locations, resulting in a lack of coherence in time series data. Traditional interpolation or completion methods struggle to capture complex dynamic relationships. Neural ordinary differential equations (NODEs) address the sparse sampling issue through continuous-time modeling, capturing more refined temporal dynamics. For example, multi-block graph neural ODE networks (MTGODEs) integrate NODEs to achieve continuity in graphs and multi-block structures. However, NODEs are limited by the dimensionality of their hidden layers, hindering their ability to model long-term dependencies. To address this, the fast-slow dynamics ODE network (GODERN), based on the theory of fast-slow dynamics, introduces fast and slow ODEs to describe complex systems at different time scales. However, it still suffers from catastrophic forgetting when modeling continuity over long timescales.

[0006] In summary, researching and developing advanced prediction methods for key parameters of industrial processes has important theoretical significance and practical application value. Summary of the Invention

[0007] In view of the defects or deficiencies of the above-mentioned prior art, the present invention provides a method for predicting key parameters of industrial processes based on spatiotemporal graph convolutional networks. First, the collected industrial process data is preprocessed and sequenced; then, the internal features of the subsequences are modeled in the variable dimension through global and local graph convolutions; then, the time modeling between subsequences is completed in the time dimension through continuous time modeling and multi-head attention, wherein the continuous time modeling is composed of multiple neural ordinary differential equations and gating mechanisms, and the continuous modeling of each subsequence in the entire time dimension is completed in sequence through a cycle; finally, the prediction of key parameters of the industrial process is realized using a multi-layer perceptron. The present invention solves the problems existing in the modeling of industrial process data, improves the prediction accuracy of key parameter data of industrial processes, and provides technical support for the optimization control of industrial processes.

[0008] The technical solution adopted by the present invention to achieve the above-mentioned purpose is: a method for predicting key parameters of industrial processes based on spatiotemporal graph convolutional networks, comprising the following steps:

[0009] S1. Collect key parameter data of industrial processes;

[0010] S2. Preprocess the data, build a data set and divide the sequence;

[0011] S3. Modeling variable dimensions within a subsequence: The features of a single sequence point within the subsequence are embedded into the spatial dimension to generate a new embedding vector, which is used by the subsequent network modeling to capture the time lag characteristics of the data. Global graph convolution is then used to model all dependencies within the subsequence. Local graph convolution is used to optimize the redundancy of the embedding vector so that the model only focuses on features relevant to the subsequence. Finally, the outputs of the two are added together to complete the modeling of the time series data within the subsequence.

[0012] S4. Temporal dimension modeling between subsequences: Continuous time modeling is used to extract finer-grained features for each subsequence to compensate for the model's lack of feature extraction due to missing data. Multi-head attention is then used to extract temporal features for the entire sequence, yielding the output features of temporal dimension modeling.

[0013] S5. Establish a multi-layer perceptron network to iteratively learn the mapping relationship between input features and output labels, thereby predicting the future sequence of key industrial process parameters;

[0014] S6. Using the data in the dataset as input, repeat steps 3 to 5 for iterative training, select the model parameters with the best accuracy as the final ideal model, and predict the future key parameter sequence based on the historical data of the industrial process.

[0015] The key parameter data of the industrial process are industrial status data and process control data collected by detection sensors and control equipment set in the industrial production environment.

[0016] The preprocessing includes removing outliers and normalizing the maximum and minimum values.

[0017] The data set is divided into training, validation, and test sets in proportion. The training set data is used for training according to a set number of iterations and then cut off. The validation set data is used to verify the accuracy of the model. The test set data is used for testing. Furthermore, each sample data in each set contains L+T sampling values, of which the first L sampling values ​​constitute the historical input data for training, and the last T sampling values ​​serve as the label value of the prediction target during model training and testing.

[0018] The sequence division is to use a sliding window of fixed size to divide the entire historical input data sequence of each sample into multiple subsequences according to a specific step size, and the length of each subsequence is equal to the size of the window.

[0019] Furthermore, the global graph convolution is to multiply the adjacency matrix A of the adaptive graph convolution by a time delay matrix π to obtain an enhanced matrix This enhances the model's perception of the time distance within the subsequence. Assuming there is an adjacency matrix A containing 3 time points, the delay matrix measures the time distance through the decay rate δ<1. The farther away from the current time step, the greater the degree of decay. Among them, the time delay matrix π is expressed in an 8-neighborhood with a step size of T as

[0020] Furthermore, the local graph convolution is to calculate the weights of neighbor nodes through local graph convolution of the adjacency matrix of the adaptive graph convolution and mask out the nodes with low weights to obtain a similarity matrix R, thereby masking the redundant information in the subsequence.

[0021] Specifically, we first define a mask matrix M to remove smaller edge values ​​in the adjacency matrix A, as shown in Formula 1:

[0022]

[0023] Where τ is the mask threshold, i and j are the indices of the matrix elements. Then use M and A to define a new similarity matrix R, where each element R ij Calculated according to the following formula:

[0024]

[0025] The continuous-time modeling includes:

[0026] The neural ordinary differential equation is used to perform continuous time modeling on the data of a single subsequence at this moment and the output of the subsequence at the previous moment to simulate the continuous evolution of the system.

[0027] After a period of modeling, the long-term dependencies in the data are captured through the gating mechanism to alleviate the forgetting problem of continuous-time modeling.

[0028] Repeat the above two steps until the time point of the next subsequence is modeled, obtain the output of the subsequence and use it for continuous time modeling of the next subsequence;

[0029] Repeat the above three steps until all subsequences are modeled.

[0030] The present invention has the following beneficial effects and advantages:

[0031] The present invention proposes a method for predicting key parameters of industrial processes based on a spatiotemporal graph convolutional network. This method, when addressing the time lag characteristics of industrial process data, extracts time lag features within the sequence through sequence partitioning and global graph convolution. Local graph convolution is used to optimize the redundancy of embedded vectors, allowing the model to focus only on features relevant to itself, mitigating the interference of redundant information. Furthermore, continuous-time modeling is used to extract finer-grained features for each subsequence, compensating for the model's insufficient feature extraction due to missing data. Overall, the present invention improves the accuracy of predictions for key parameters of industrial processes based on a spatiotemporal graph convolutional network. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] Figure 1 Flow chart of the method of the present invention.

[0033] Figure 2 The figure is a flow chart of the intra-subsequence and inter-subsequence modeling methods of the present invention.

[0034] Figure 3 Schematic diagram of the delay matrix in the global graph convolution of the present invention.

[0035] Figure 4 This is a flow chart of the continuous-time modeling method of the present invention. DETAILED DESCRIPTION

[0036] To make the above-mentioned objects, features, and advantages of the present invention more readily apparent, the specific implementation methods of the present invention are described in detail below with reference to the accompanying drawings. The following description sets forth many specific details to facilitate a full understanding of the present invention. However, the present invention can be implemented in many other ways than those described herein, and those skilled in the art may make similar modifications without departing from the scope of the invention. Therefore, the present invention is not limited to the specific implementation methods disclosed below.

[0037] Unless otherwise defined, all technical and scientific terms used herein have the same meanings as commonly understood by those skilled in the art to which this invention belongs. The terms used in this specification are for the purpose of describing specific embodiments only and are not intended to limit the present invention. The present invention will be further described in detail below with reference to the accompanying drawings and examples.

[0038] like Figure 1 The flowchart of the method of the present invention is shown in FIG. A method for predicting key parameters of industrial processes based on a spatiotemporal graph convolutional network. The programming language used in the program execution steps of the present invention is not limited to MATLAB, Python, etc.

[0039] The specific steps of the present invention are as follows:

[0040] Step 1: Collect key industrial parameter data (taking the loosening and rehydration process in the food processing industry as an example, the variable data includes 24 variables, including the moisture content of the material at the drum inlet and the moisture content of the material at the drum outlet, the actual water addition value, the water valve opening, the actual outlet temperature, the water addition set value, the actual temperature of bucket 1, the actual temperature of bucket 2, and the water addition coefficient. The key parameter is the moisture content of the material at the drum outlet). The present invention can predict the moisture content of the material at the drum outlet based on these 24 variables, providing a reference for industrial control.

[0041] Step 2: Remove outliers from each data according to the set error range, and then perform maximum and minimum normalization processing to normalize the data to between (0, 1).

[0042] Step 3: Create a sample data set for model training and testing. Each sample contains L+T sampling values, where the first L sampling values ​​constitute the historical input data for training, and the last T sampling values ​​serve as prediction targets, that is, the label values ​​during model training and testing.

[0043] Step 4: Use a fixed-size sliding window to divide the sample data generated in step 3 into multiple subsequences according to a specific step size. The length of each subsequence is equal to the size of the window.

[0044] Step 5: Model the variable dimensions within the subsequence as follows Figure 2 As shown in the figure, the data after data partitioning is then subjected to global graph convolution to extract all dependencies within the entire subsequence. By flexibly modeling the graph structure, it can more effectively extract and utilize global features in the data. Furthermore, the data after data partitioning is also subjected to local graph convolution to calculate the weights of neighboring nodes and mask out nodes with low weights, thereby alleviating the problem of information redundancy in industrial process data.

[0045] Furthermore, the global graph convolution is based on the adaptive graph convolution, where the adjacency matrix A is multiplied by a time delay matrix π to obtain an enhanced matrix This enhances the model's perception of the time distance within the subsequence. Assume that there is an adjacency matrix A containing 3 time points, then the delay matrix is ​​as follows Figure 3 As shown, the matrix measures the time distance by the decay rate δ<1. Specifically, the farther away from the current time step, the greater the degree of decay. Among them, the time delay matrix π is in an 8-neighborhood with a step size of T.

[0046]

[0047] Furthermore, the local graph convolution is based on adaptive graph convolution, replacing its adjacency matrix with a similarity matrix to mask the redundant information within the subsequence. Local graph convolution is used to calculate the weights of neighboring nodes and mask out nodes with low weights. Specifically, we first define a mask matrix M to remove small edge values ​​in the adjacency matrix A, as shown in Formula 1:

[0048]

[0049] Where τ is the mask threshold, i and j are the indices of the matrix elements. Then use M and A to define a new similarity matrix R, where each element R ij Calculated according to the following formula:

[0050]

[0051] Step 6: Model the time dimension between subsequences as follows Figure 2 As shown in the figure. For each subsequence, continuous time modeling is used to extract finer-grained features, which makes up for the problem of insufficient feature extraction caused by missing data. Then, multi-head attention is used to extract the time features of the entire sequence, and the output of time dimension modeling is obtained. The specific process of continuous time modeling is as follows: Figure 4 shown.

[0052] Step 6-1: Use the Neural Ordinary Differential Equation to perform continuous-time modeling on the data of a single subsequence at this moment and the output of the subsequence at the previous moment to simulate the continuous evolution of the system;

[0053] Step 6-2: After a period of modeling, capture long-term dependencies in the data through a gating mechanism to alleviate the forgetting problem of continuous-time modeling.

[0054] Step 6-3: Repeat steps 6-1 and 6-2 until the time point of the next subsequence is modeled, obtain the output of the subsequence and use it for continuous time modeling of the next subsequence;

[0055] Step 6-4: Repeat step 6-3 above until all subsequences are modeled.

[0056] Step 6-5: Input all the above subsequences into the multi-head attention to extract the temporal dependency of the entire sequence from a global perspective.

[0057] Step 7: Use the extracted features combined with the multi-layer perceptron to predict the future key parameter sequence of the industrial process.

[0058] Step 8: Repeat steps 4, 5, 6, and 7, select the model parameters with the best accuracy as the final optimization model, and predict the future key parameter sequence based on the historical data of the industrial process.

[0059] The data set is a loose rehydration process of a food processing industry. The historical data step size L is 16, the prediction data step size is 6, and the evaluation indicators are mean absolute error (MAE), root mean square error (RMSE) and weighted absolute percentage error (WAPE). The performance results of the proposed method and the other seven methods are shown in Table 1, which shows that the proposed method improves the prediction accuracy of key parameter data of the industrial process compared with other existing methods, providing technical support for the optimization control of the industrial process.

[0060] Table 1

[0061]

[0062] The above description is only a preferred embodiment of the present invention and does not limit the present invention in any way. Any simple modification, change and equivalent structural change made to the above embodiment based on the technical essence of the present invention shall still fall within the scope of protection of the technical solution of the present invention.

Claims

1. A method for predicting key parameters of industrial processes based on spatiotemporal graph convolutional networks, characterized by: The steps include: S1. Collect key parameter data of industrial processes; the key parameter data of industrial processes are industrial status data and process control data collected by detection sensors and control equipment set in the industrial production environment; S2. Preprocess the data, build a data set and divide the sequence; S3. Modeling variable dimensions within a subsequence: The features of a single sequence point within the subsequence are embedded into the spatial dimension to generate a new embedding vector, which is used by the subsequent network to capture the time lag characteristics in the data during modeling. Global graph convolution is then used to model all dependencies within the subsequence. Local graph convolution is used to optimize the redundancy of the embedding vector so that the model only focuses on features related to itself. Finally, the outputs of the two are added together to complete the modeling of the temporal data within the subsequence. The local graph convolution is to calculate the weights of neighbor nodes through the adjacency matrix of the adaptive graph convolution and mask out nodes with low weights to obtain a similarity matrix R, thereby masking redundant information within the subsequence. S4. Temporal dimension modeling between subsequences: Continuous time modeling is used to extract finer-grained features for each subsequence to compensate for the model's lack of feature extraction due to missing data. Multi-head attention is then used to extract temporal features for the entire sequence, yielding the output features of temporal dimension modeling. The continuous-time modeling includes: The neural ordinary differential equation is used to perform continuous time modeling on the data of a single subsequence at the current moment and the output of the subsequence at the previous moment to simulate the continuous evolution of the system. After a period of modeling, the long-term dependencies in the data are captured through the gating mechanism to alleviate the forgetting problem of continuous-time modeling. Repeat the above two steps until the time point of the next subsequence is modeled, obtain the output of the subsequence and use it for continuous time modeling of the next subsequence; Repeat the above three steps until all subsequences are modeled; S5. Establish a multi-layer perceptron network to iteratively learn the mapping relationship between input features and output labels, thereby predicting the future sequence of key industrial process parameters; S6. Using the data in the dataset as input, repeat steps 3 to 5 for iterative training, select the model parameters with the best accuracy as the final ideal model, and predict the future key parameter sequence based on the historical data of the industrial process.

2. The method for predicting key parameters of industrial processes based on spatiotemporal graph convolutional networks according to claim 1 is characterized in that: The preprocessing includes removing outliers and normalizing the maximum and minimum values.

3. The method for predicting key parameters of industrial processes based on spatiotemporal graph convolutional networks according to claim 1 is characterized in that: The data set is divided into training set, validation set and test set in proportion; the training set data is used for training and cutoff according to the set number of iterations; the validation set data is used to verify the accuracy of the model; the test set data is used for testing; further, each sample data in each set contains L+T sampling values, of which the first L sampling values ​​constitute the historical input data for training, and the last T sampling values ​​serve as the label value of the prediction target during the model training and testing process.

4. The method for predicting key parameters of industrial processes based on spatiotemporal graph convolutional networks according to claim 1 is characterized in that: The sequence division is to use a sliding window of fixed size to divide the entire historical input data sequence of each sample into multiple subsequences according to a specific step size, and the length of each subsequence is equal to the size of the window.

5. The method for predicting key parameters of industrial processes based on spatiotemporal graph convolutional networks according to claim 1 is characterized in that: The global graph convolution is to multiply the adjacency matrix A of the adaptive graph convolution by a time delay matrix Get the enhanced matrix .

Images