Prediction method for irregular time series data in process industry

Abstract

The invention provides a prediction method for irregular time series data in the process industry, and belongs to the technical field of industrial time series data prediction, and the method comprises the following steps: data collection and preprocessing; constructing graph representation of the irregular time series data; constructing and training a space-time relation graph convolutional neural network model; according to the method, a space-time relational graph convolutional neural network model is established, online prediction and practical application are carried out, and the space-time relational graph convolutional neural network model comprises a full-connection graph conversion module, a space-time relational graph convolutional layer and a layered sandwich structure. And high-precision prediction of an asynchronous and complex space-time dependent structure is realized based on a space-time relational graph convolution module, a hierarchical sandwich strategy further ensures calculation efficiency and model performance on large-scale graph data, and the method is suitable for the requirements of real-time prediction, fault early warning and the like of a large number of asynchronous sensor measuring points in an industrial field.

Description

Technical Field

[0001] The present invention belongs to the technical field of industrial time series data prediction, and particularly relates to a prediction method for irregular time series data in process industry. Background Art

[0002] For the prediction method of irregular time series data in process industry, it is necessary to combine the dynamic characteristics of time series data and the special requirements of industrial scenarios (such as high noise, multivariable coupling, physical constraints, etc.). Its data has the following characteristics: (1) Irregularity: The data sampling interval is not fixed (such as event-triggered acquisition, sensor failure, manual recording, etc.); (2) Multivariable coupling: There are strong physical / chemical correlations between process industry variables (such as temperature, pressure, flow rate, etc.); (3) Noise and missing: Sensor noise, data loss, and outliers are common; (4) Long-term and short-term dependencies: It is necessary to capture both short-term fluctuations (such as equipment status) and long-term trends (such as process degradation).

[0003] Time series prediction technology has wide application potential in process industry (such as chemical industry, refining industry). Predicting future sequences based on historical data can not only achieve early monitoring and warning of processes, but also be widely applied in soft sensors and digital twin systems.

[0004] Irregular time series data refers to a time series with an unfixed sampling interval and asynchronous characteristics of sampling points between variables, such as Figure 1 shown as Figure 1 (a) shows that the regular time series has a fixed sampling interval and the sampling points of different variables v are aligned in time. Figure 1 (b) shows the characteristics of the irregular time series. This kind of data is widespread in fields such as industry, medical care, and finance. Among them, the variables have irregular sampling intervals, and different variables have different sampling frequencies and misaligned timestamps. Compared with regular time series, irregular time series have more complex and variable spatio-temporal structure characteristics. However, most of the current time series prediction methods are based on the assumption of regular sampling. For example, the multi-dimensional spatio-temporal attention method proposed in patent CN115034325B and the process industry multi-variable time series pre-training method proposed in patent CN117312851A do not consider the problem of irregular sampling.

[0005] In recent years, some processing methods for irregular time series have emerged one after another. For example, Patent CN115329166A proposes a classification method applicable to irregular time series data, but this method focuses on the classification task and does not involve the prediction problem concerned by the present invention. In addition, Patent CN118194242A and Patent CN118245474B both propose specific methods for irregular time series prediction tasks, but they both adopt the strategy of data pre-alignment, that is, expanding univariate observations to cover all observed timestamps. This strategy leads to an exponential increase in the scale of the input data matrix as the number of variables grows, resulting in a significant increase in memory occupancy and computational cost during the training and inference stages.

[0006] In contrast, the method proposed by the present invention no longer relies on data pre-alignment, but models the irregular time series as a fully connected graph structure, where nodes represent actual observation points and edges represent the potential spatio-temporal relationships between observation points. This graph representation method avoids the problem of exponential increase in the scale of the input matrix as the number of variables increases. In addition, through the representation learning mechanism of the graph neural network, this method can naturally and efficiently handle different sampling frequencies and unaligned timestamps between variables, directly capture the complex asynchronous spatio-temporal dependencies in the data, significantly reduce the computational complexity, and improve the prediction performance for industrial irregular time series data. Summary of the Invention

[0007] Aiming at the deficiencies of the prior art, the present invention proposes a prediction method for irregular time series data in the process industry, constructs a fully connected graph representation without pre-alignment for irregularly sampled multi-variable time series data in the process industry environment, and realizes high-precision prediction of asynchronous and complex spatio-temporal dependence structures based on the spatio-temporal relationship graph convolution module.

[0008] The present invention adopts the following technical solutions to solve the above problems:

[0009] A prediction method for irregular time series data in the process industry, the method comprising the following steps:

[0010] S1: Data collection and preprocessing: Deploy N sensors in the required scenario to monitor different process parameters, preprocess the collected data, and eliminate or correct extreme outliers or obvious measurement errors;

[0011] S2: Construct a graph representation of irregular time series data: Consider all historical observation points and future time points to be predicted as nodes in a fully connected graph, so as to flexibly model the relationship between irregular time series using a graph neural network;

[0012] S3: Construct and train a spatio-temporal relational graph convolutional neural network model: Introduce a spatio-temporal relational graph convolutional network (STRGCN) on a fully connected graph to naturally capture asynchronous spatio-temporal dependencies. Based on the spatio-temporal relational graph convolutional operator improved from relational graph convolution, after completing the iteration of the spatio-temporal relational graph convolutional structure, decode the nodes to be predicted in the future and perform regression prediction;

[0013] S4: Online prediction and practical application: After obtaining real-time sensor data and appending it to the timestamp nodes of the corresponding variables in the graph, perform online measurement. The system can use the prediction results for various decision-making scenarios such as process control, anomaly warning, energy consumption optimization, or soft sensor output.

[0014] Further, in S1, denote the historical observation set of the nth sensor as where t n,i represents the ith observation timestamp of this sensor, and z n,i represents the corresponding observed value; K n is the total number of data points observed by this sensor during the historical period. The historical observations of all sensors can be combined and denoted as If there are obvious noises or outliers in the data, no interpolation or alignment operations are required to retain the asynchronous time series information of the original data and provide as many effective features as possible for subsequent modeling.

[0015] Further, in S2, for the nth sensor, in addition to the historical observation set in S1, another set of future timestamps is defined for prediction. The set of all time points of this sensor can be expressed as

[0016]

[0017] The corresponding node features can be written as:

[0018]

[0019] where z unk is used as a placeholder and is learned or overwritten by the model itself during the prediction process; each node also carries a variable identifier to distinguish nodes from different sensors (variables); combine all the above nodes (historical and future) to obtain the total number of nodes and establish a directed edge for any node pair (i, j) to form a fully connected graph;

[0020] Denote as the original feature matrix of all nodes (where C is the original feature dimension of a single node), as the timestamp vector corresponding to all nodes, as the variable identifier vector corresponding to all nodes;

[0021] Define the conversion function Convert the irregular time series and the set of future timestamps into

[0022] Furthermore, in S3, to reduce the computational complexity, a "sandwich"-type hierarchical aggregation structure is constructed. The resolution of the graph is reduced through the "supernode" mechanism, and then local-global information fusion is performed. The specific steps are as follows:

[0023] S301: Configure parameters and construct the spatio-temporal relationship graph convolutional neural network model structure;

[0024] S302: Generate supernodes and perform "sandwich"-type hierarchical convolution;

[0025] S303: Train the model through the decoder and loss function to ensure that the model can effectively learn the spatio-temporal interaction rules in the irregular time series data.

[0026] Furthermore, in S301, the parameters include the initial node features, time encoding, and variable identity encoding. To highlight the weakening of the dependence degree of nodes with a relatively large time distance, the present invention adopts normalization with a time factor during aggregation. Denote a i * ,j as the normalization coefficient for node j to propagate messages to node i, which can be defined as follows:

[0027]

[0028] where e r is the learnable scalar weight associated with the variable pair r (i.e., in a certain relationship), which is used to balance the relative influence between different variables; p i , p j are the time embedding vectors of nodes i and j respectively. After combining the above decoupling and normalization strategies, the update of node i at the k-th layer can be written as

[0029]

[0030] where is the hidden vector of node i at the k-th layer, is the transformation matrix corresponding to the relationship r, σ(·) is the activation function, is the self-connection transformation matrix. The present invention abstracts the entire layer operation as

[0031] H (k+1) =STRGCNLayer(H (k) ,P,S,Θ (k) )

[0032] where H (k) is the hidden vector of the k-th layer, P is the time encoding, and S is the spatial encoding Θ (k) contains all the learnable parameters of this layer (such as Q, φ(·), e r , W0), Q is a learnable matrix, and multiple layers stacked can gradually refine higher-order spatio-temporal correlation information.

[0033] Furthermore, in S302, the original nodes are segmented or sampled according to the time axis, and a batch of supernodes are aggregated Each supernode can represent a unified abstraction within a certain time range (or several adjacent nodes). Let their spatio-temporal encodings be denoted as P β and S β respectively. Convolution is performed on the supernodes as a unit in the middle layer to reduce the graph size.

[0034] Furthermore, in S302, the "sandwich"-type hierarchical convolution includes the following steps:

[0035] When updating the k-th layer, first perform a spatio-temporal relationship graph convolution propagation on all the original nodes and supernodes at the bottom layer to obtain the supernode representation H β,(k) ; then perform convolution update on itself in the middle layer to obtain H′ β,(k) ; finally, map the information of back to the original node resolution at the top layer to generate H′ (k+1) . The three steps are recorded as follows:

[0036] H β,(k) = STRGCNLayer(H (k) , P, S, Θ bottom,(k) )

[0037] H′ β,(k) = STRGCNLayer(H β,(k) , P β , S β , Θ middle,(k) )

[0038] H′ (k+1) = STRGCNLayer(H′ β,(k) , P β , S β , Θ top,(k) )

[0039] Finally, the residual mechanism is adopted

[0040] H (k+1) = H′ (k+1) + H (k)

[0041] While retaining the underlying information and incorporating the global context of the super-node layer, this process can be regarded as a composite layer that can complete multi-granularity aggregation of the entire graph with less computational cost.

[0042] Further, in S303, the following steps are included:

[0043] (1) Prediction task and decoding

[0044] After completing the iteration of the spatio-temporal relationship graph convolution structure, the final representation of each node is obtained. For the nodes to be predicted in the future, their corresponding representations can be mapped to real values through a decoder (multi-layer perceptron MLP):

[0045]

[0046] Where has the same dimension as the real observation z n,j and is used for regression prediction.

[0047] (2) Loss function

[0048] For the regression scenario, mean squared error (MSE) or mean absolute error (MAE) can be adopted. Taking MSE as an example:

[0049]

[0050] Where is the total number of future timestamps, is the model prediction value, and z n,j is the real value. If it is necessary to balance the model complexity, L2 regularization can be introduced:

[0051]

[0052] Θ contains all the trainable parameters of the model, and λ is the regularization coefficient, which is used to prevent overfitting.

[0053] (3) Training process

[0054] Divide the historical observations into a training set and a validation set, and iteratively minimize on the training set through stochastic gradient descent (SGD) or adaptive optimization algorithm (Adam). Monitor the error decline trend on the validation set. If the convergence condition or the maximum number of iterations is reached, stop training. At this time, the model can effectively learn the spatio-temporal interaction rules in irregular time-series data.

[0055] Further, in S4, the following steps are included:

[0056] S401: Construct new nodes and update the graph: For the newly arrived real-time observations (tnew , z new ), and according to the variable index n, it is regarded as a new node and added to to obtain an updated (X, T, S) matrix.

[0057] S402: Forward inference: Input the updated node features into the trained spatio-temporal relationship graph convolutional model, and the prediction results for the current or future time stamps can be obtained by combining the convolutional calculations of each layer

[0058] S403: Output prediction value and decision: The system can use the prediction results for various decision-making scenarios such as process control, anomaly warning, energy consumption optimization, or soft sensor output. Since the data is not interpolated or pre-aligned in the present invention, the model can still ensure efficient update and inference when new data arrives.

[0059] Furthermore, the sensors are used to monitor key process variables including but not limited to temperature, flow rate, and pressure.

[0060] Advantages of the present invention:

[0061] 1. The present invention constructs a fully connected graph representation without pre-alignment for irregularly sampled multi-variable time series data in a process industrial environment, and realizes high-precision prediction of asynchronous and complex spatio-temporal dependence structures based on the spatio-temporal relationship graph convolutional module.

[0062] 2. The hierarchical "sandwich" strategy further ensures computational efficiency and model performance on large-scale graph data, and is suitable for real-time prediction and fault warning requirements in industrial sites with a large number of asynchronous sensor measurement points. Description of the drawings

[0063] In order to more clearly illustrate the specific embodiments of the present invention, the following will briefly introduce the drawings required for the description of the specific embodiments. Obviously, the following drawings are some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0064] Figure 1 is a comparison diagram of regular time series and irregular time series in the background art;

[0065] Figure 2 is a flowchart of the present method;

[0066] Figure 3 is a schematic diagram of the spatio-temporal relationship graph convolutional neural network model proposed by the present method;

[0067] Figure 4 is a schematic diagram of super-node generation and hierarchical sandwich structure. Detailed implementation manners

[0068] The following describes the detailed implementation manners of the present invention to facilitate those skilled in the art of the present technology to understand the present invention. However, it should be clear that the present invention is not limited to the scope of the detailed implementation manners. For those of ordinary skill in the art of the present technology, various changes are obvious within the spirit and scope of the present invention defined and determined by the appended claims. All inventions created using the concept of the present invention are within the scope of protection.

[0069] It should be noted that the professional terms used in the present invention are only for the purpose of describing specific embodiments and are not intended to limit the protection scope of the present invention. Unless otherwise specifically stated, various instruments and devices used in the following embodiments of the present invention can be obtained through market purchase or prepared by existing methods.

[0070] As Figure 3 shown, the present solution constructs an overview of the proposed spatio-temporal relationship graph convolutional neural network model, including the following key components: (a) a fully connected graph conversion module that converts irregular time series data into a fully connected graph with a compact representation; (b) a spatio-temporal relationship graph convolutional layer that aims to capture asynchronous spatio-temporal dependencies; (c) a hierarchical sandwich structure that integrates local and global semantic relationships while alleviating computational complexity.

[0071] Embodiment 1

[0072] As Figure 2 shown, the present solution provides a prediction method for irregular time series data in process industries, specifically including the following steps:

[0073] S1: Data acquisition and preprocessing.

[0074] Deploy N sensors in the required scenario to monitor different process parameters, and preprocess the collected data to eliminate or correct extreme outliers or obvious incorrect measurements.

[0075] In the production process of process industries (such as chemical industry, oil refining, etc.), multiple sensors are usually deployed to monitor key process variables (such as temperature, pressure, flow rate, etc.). Due to differences in the hardware characteristics or sampling strategies of each sensor, the obtained time series data often has the characteristics of: non-fixed sampling intervals (the sampling time intervals of the same variable are not equal) and asynchronous sampling between variables (the observation timestamps of different variables are not aligned and may have a large time deviation). Therefore, usually deploy N sensors (which can also be denoted as variables v1,..., vN) to monitor different process parameters, such as temperature, flow rate, pressure, etc. Due to differences in the hardware characteristics or sampling strategies of each sensor, the sampling intervals of each are not fixed, and the observation timestamps between different sensors are often not aligned.

[0076] Let the historical observation set of the n-th sensor be denoted as where t n,i represents the i-th observation timestamp of the sensor, and z n,i represents the corresponding observation value; K n is the total number of data points observed by the sensor during the historical period. The historical observations of all sensors can be combined and denoted as If there are obvious noises or outliers in the data, no interpolation or alignment operations are required to retain the asynchronous time series information of the original data and provide as many effective features as possible for subsequent modeling.

[0077] S2: Construct a graph representation of the irregular time series data.

[0078] All historical observation points and the time points to be predicted in the future are uniformly regarded as nodes in a fully connected graph, so as to flexibly model the relationships between irregular time series using a graph neural network.

[0079] For the n-th sensor, in addition to the historical observation set in S1, another set of future timestamps is defined for prediction. The set of all time points of this sensor can be expressed as

[0080]

[0081] The corresponding node features can be written as:

[0082]

[0083] where z unk is used as a placeholder and is learned or overwritten by the model during the prediction process; each node also carries a variable identifier to distinguish nodes from different sensors (variables); all the above nodes (historical and future) are combined to obtain the total number of nodes , and a directed edge is established for any node pair (i, j) to form a fully connected graph;

[0084] Denote as the original feature matrix of all nodes (where C is the original feature dimension of a single node), as the time stamp vector corresponding to all nodes, as the variable identifier vector corresponding to all nodes;

[0085] Define the transformation function to map the irregular time series and the set of future timestamps to

[0086] In this way, even if the sampling frequencies and time distributions of different sensors are different, they can be uniformly processed under the same graph structure.

[0087] S3: Construct and train a spatio-temporal relationship graph convolutional neural network model.

[0088] Introduce a spatio-temporal relationship graph convolutional network (STRGCN) on the fully connected graph to naturally capture asynchronous spatio-temporal dependencies. Based on the spatio-temporal relationship graph convolutional operator improved from the relational graph convolution, after completing the iteration of the spatio-temporal relationship graph convolution structure, decode the future nodes to be predicted and perform regression prediction.

[0089] Optionally, as Figure 4 shown, the computational overhead can be effectively reduced on the large-scale fully connected graph through a hierarchical "sandwich" structure. Figure 4 (a) shows generating supernodes for each variable by uniformly sampling nodes along the time axis, with the window length predefined. Figure 4 (b) provides an intuitive understanding of the bottom, middle, and top layers in the sandwich structure.

[0090] Specifically, it includes the following steps:

[0091] S301: Configure parameters and construct the spatio-temporal relationship graph convolutional neural network model structure.

[0092] The parameters include initial node features, time encoding, and variable identity encoding. To highlight the weakening of the dependence degree of nodes with a relatively long time distance, the present invention adopts normalization with a time factor during aggregation. Denote as the normalization coefficient for node j to propagate messages to node i, which can be defined as follows:

[0093]

[0094] where e r is a learnable scalar weight associated with the variable pair r (i.e., a certain relationship in), used to balance the relative influence between different variables; p i , p j are the time embedding vectors of nodes i and j respectively. After combining the above decoupling and normalization strategies, the update of node i at the k-th layer can be written as

[0095]

[0096] where is the hidden vector of node i at the k-th layer, is the transformation matrix corresponding to the relationship r, σ(·) is the activation function, is the self-connection transformation matrix. The present invention abstracts the entire layer operation as

[0097] H (k+1) = STRGCNLayer(H (k) , P, S, Θ(k) )

[0098] where H (k) is the hidden vector of the k-th layer, P is the temporal encoding, S is the spatial encoding Θ (k) contains all learnable parameters of this layer (such as Q, φ(·), e r , W0), Q is a learnable matrix, and multiple layers stacked can gradually refine higher-order spatio-temporal correlation information.

[0099] S302: Generate supernodes and perform "sandwich"-style hierarchical convolution.

[0100] (1) Supernode generation: As Figure 3 (a) shows, segment or sample the original nodes according to the time axis and aggregate to obtain a batch of supernodes Each supernode can represent a unified abstraction within a certain time range (or several adjacent nodes), and let their spatio-temporal encodings be denoted as P β and S β , respectively. Perform convolution on the intermediate layer with supernodes as the unit to reduce the graph size.

[0101] (2) Sandwich-style hierarchical convolution:

[0102] As Figure 3 (b) shows, when updating the k-th layer, first perform a spatio-temporal relationship graph convolution propagation on all original nodes and supernodes at the bottom layer to obtain the supernode representation H β,(k) ; then perform convolution update on itself at the intermediate layer to obtain H′ β,(k) ; finally, map the information of back to the original node resolution at the top layer to generate H′ (k+1) . Denote the three steps as follows:

[0103] H β,(k) = STRGCNLayer(H (k) , P, S, Θ bottom,(k) )

[0104] H′ β,(k) = STRGCNLayer(H β,(k) , P β , S β , Θ middle,(k) )

[0105] H′ (k+1) = STRGCNLayer(H′ β,(k) , P β , S β , Θ top,(k) )

[0106] Finally, adopt the residual mechanism

[0107] H (k+1) = H′ (k+1) + H (k)

[0108] While retaining the underlying information and incorporating the global context of the super-node layer, this process can be regarded as a composite layer that can complete multi-granularity aggregation of the entire graph at a relatively low computational cost.

[0109] S303: Train the model through the decoder and loss function to ensure that the model can effectively learn the spatio-temporal interaction patterns in irregular time-series data.

[0110] (1) Prediction task and decoding

[0111] After completing the iteration of the spatio-temporal relationship graph convolution structure, the final representation of each node is obtained. For the nodes to be predicted in the future, their corresponding representations can be mapped to real values through the decoder (multi-layer perceptron MLP):

[0112]

[0113] where has the same dimension as the real observation z n,j and is used for regression prediction.

[0114] (2) Loss function

[0115] For the regression scenario, mean squared error (MSE) or mean absolute error (MAE) can be adopted. Taking MSE as an example:

[0116]

[0117] where is the total number of future timestamps, is the model prediction value, and z n,j is the real value. If it is necessary to balance the model complexity, L2 regularization can be introduced:

[0118]

[0119] Θ contains all the trainable parameters of the model, and λ is the regularization coefficient, which is used to prevent overfitting.

[0120] (3) Training process

[0121] Divide the historical observations into a training set and a validation set, and iteratively minimize on the training set through stochastic gradient descent (SGD) or adaptive optimization algorithm (Adam) Monitor the error decline trend on the validation set and stop training if the convergence condition is reached or the maximum number of iterations is reached. At this time, the model can effectively learn the spatial-temporal interaction rules in the irregular time-series data.

[0122] S4: Online prediction and practical application.

[0123] After obtaining the real-time sensor data and appending it to the timestamp nodes of the corresponding variables in the graph, perform online measurement. The system can use the prediction results for various decision-making scenarios such as process control, anomaly warning, energy consumption optimization, or soft sensor output.

[0124] Specifically, it includes the following steps:

[0125] S401: Construct new nodes and update the graph: For the newly arrived real-time observation (t new , z new ), according to the variable index n, regard it as a new node and add it to to obtain the updated (X, T, S) matrix.

[0126] S402: Forward inference: Input the updated node features into the trained spatio-temporal relationship graph convolutional model, and the prediction results for the current or future timestamps can be obtained by combining the convolutional calculations of each layer.

[0127] S403: Output prediction values and make decisions: The system can use the prediction results for various decision-making scenarios such as process control, anomaly warning, energy consumption optimization, or soft sensor output. Since the data is not interpolated or pre-aligned in this invention, the model can still ensure efficient update and inference when new data arrives.

[0128] Through the above steps, the present invention constructs a fully connected graph representation without pre-alignment for irregularly sampled multi-variable time-series data in the process industry environment, and realizes high-precision prediction of asynchronous and complex spatio-temporal dependence structures based on the spatio-temporal relationship graph convolutional module. The hierarchical "sandwich" strategy further ensures the computational efficiency and model performance on large-scale graph data, and is suitable for real-time prediction and fault warning requirements in industrial sites with a large number of asynchronous sensor measurement points.

[0129] Embodiment 2

[0130] As a preferred embodiment of the present invention, the following embodiments aim to illustrate the specific implementation methods and experimental evaluation results of the present invention in the prediction task of irregular multi-variable time-series data, so as to help those skilled in the art understand and apply the technical solution more deeply. It should be emphasized that the data sources, model parameters, and index results given in this embodiment are not the only limitations of the present invention, and the relevant details can be flexibly adjusted according to actual needs. Specifically, it includes the following steps:

[0131] 1. Dataset and Experimental Settings.

[0132] (1) Data Source

[0133] This embodiment is verified on a representative real process industrial irregular multivariate time series dataset DIST-Irr. DIST-Irr is collected from the sensor data of the dual-column distillation in a chemical plant located in China. The DIST-Irr dataset is derived from the methanol distillation process in the chemical plant, covering the distributed control system data from July 1st to August 1st, 2023. The dataset contains 37 sensors (variables) and consists of 133,920 data points, ensuring high-resolution data collection.

[0134] For each of the above datasets, it is randomly divided into a training set, a validation set, and a test set in the ratio of 60%:20%:20%. Denote that the dataset contains 37 sensors (variables), and the historical observation sets of each variable The set of timestamps to be predicted in the future

[0135] (2) Experimental Environment and Settings

[0136] Training and inference are carried out on a GPU server equipped with an NVIDIA GeForce RTX 4090 (24GB) video memory; the operating system is Linux, and the deep learning framework uses PyTorch.

[0137] For the DIST-irr dataset: Use the observations in the past 1 hour (multivariate irregular time series) to predict the values of each variable in the next hour.

[0138] 2. Specific Parameter Configuration

[0139] Consistent with the symbols in Embodiment 1, the dimensions of node features, time encoding, and variable encoding, as well as the key hyperparameters in the hierarchical "sandwich" structure, are given here.

[0140] (1) Initial Feature and Time Encoding Dimensions

[0141] Denote the input dimension C of the original node feature matrix varies depending on the dataset and is obtained through linear mapping to get

[0142]

[0143] In this embodiment, D = 64.

[0144] Timestamp vector is obtained through the continuous time encoding function P = TimeEmbed(T) The time encoding method can adopt sine-cosine basis functions or learnable embeddings. In this embodiment, a sine-cosine hybrid type is adopted.

[0145] (2) Construction and training of the spatio-temporal relationship graph convolutional network

[0146] Low-rank decomposition matrix Q: Corresponding to the learnable matrix in used to capture the time difference between nodes i and j.

[0147] Variable embedding function φ(·): Each sensor variable identifier is mapped to a 64-dimensional vector φ(s i ). Then it is used together with the time embedding to calculate

[0148] Normalization coefficient: Let

[0149]

[0150] where e r is the scalar weight for a certain variable pair r, both of which are learned during training.

[0151] (3) Hierarchical "sandwich" structure

[0152] Hierarchical supernodes: The original nodes are aggregated into several "supernodes" in chronological order. In this embodiment, the number of supernodes varies according to the dataset to construct a smaller graph in the middle layer and reduce the computational overhead.

[0153] Three-layer update: First, perform a spatio-temporal relationship graph convolutional propagation on all the original nodes and supernodes at the bottom layer to obtain the supernode representation H β,(k) ; then perform a convolutional update on itself at the middle layer to obtain H′ β,(k) ; finally, map the information of back to the original node resolution at the top layer to generate H′ (k+1) . The three steps are recorded as follows:

[0154] H β,(k) =STRGCNLayer(H (k) ,P,S,Θ bottom,(k) )

[0155] H′ β,(k) =STRGCNLayer(H β,(k) ,P β ,S β ,Θ middle,(k) )

[0156] H′ (k+1) =STRGCNLayer(H′ β,(k),P β ,S β ,Θ top,(k) )

[0157] Finally, the residual mechanism is adopted

[0158] H (k+1) = H' (k+1) + H (k)

[0159] It not only retains the underlying information but also incorporates the global context of the supernode layer. This process can be regarded as a composite layer, which can complete the multi-granularity aggregation of the entire graph with less computational cost.

[0160] (4) Training Loss and Hyperparameters

[0161] [[ID=2�]]The loss function uses the root mean square error MSE:

[0162]

[0163] Among them, λ‖Θ‖ 2 λ takes 10 in -3 .

[0164] The optimizer selects Adam, and the initial learning rate is 1×10 -3 , and the batch size is set to 32.

[0165] The upper limit of the number of training rounds is 30 rounds, and it is observed on the validation set and stopped early after 5 rounds without further decrease.

[0166] 3. Experimental Results and Beneficial Effects

[0167] Baseline method: The most advanced model t-PatchGNN in the field of irregular time series analysis is selected as the baseline model for comparison, and some key components are removed from the spatio-temporal relational graph convolution (STRGCN) model proposed in this patent to verify the effectiveness of the proposed scheme, including re-GCN (replacing the proposed spatio-temporal relational graph convolution operator with a traditional graph convolution operator) and w / o-Sandwich (removing the hierarchical sandwich structure).

[0168] Table 1 shows the MSE experimental results of the model on the DIST-Irr multivariate time series prediction dataset (the lower the MSE index, the better):

[0169] Table 1: Implementation experimental results of the irregular multivariate time series prediction task

[0170]

[0171] The experimental results show that the spatio-temporal relational graph convolutional model STRGCN proposed in this patent has achieved the optimal prediction performance. In addition, compared with t-PatchGNN that adopts a patch-based time segmentation strategy, STRGCN can capture asynchronous time relationships at the node level and local fine-grained level, thus achieving a reduction of about 12% in MSE. After removing the key innovations of this patent, namely, the spatio-temporal relational graph convolutional operator and the hierarchical sandwich structure, the prediction performance has declined, demonstrating the beneficial effects of the content proposed in this patent.

[0172] The above has described this embodiment in detail through examples, but the content is only the preferred embodiment of this embodiment and cannot be considered as limiting the implementation scope of this embodiment; all equivalent changes and improvements made according to the scope of this embodiment application shall still fall within the patent coverage scope of this embodiment.

Claims

1. A prediction method for irregular time-series data in the process industry, characterized in that: The method includes the following steps: S1: Data collection and preprocessing: Deploy N sensors in the required scenario to monitor different process parameters. The collected data is preprocessed, and extreme abnormal points or obvious measurement errors are removed or corrected. S2: Constructing a graph representation of irregular time-series data: All historical observation points and future time points to be predicted are uniformly regarded as nodes in a fully connected graph, so as to flexibly model the relationships between irregular time series using a graph neural network. S3: Constructing and training a spatio-temporal relationship graph convolutional neural network model: Introduce a spatio-temporal relationship graph convolutional network (STRGCN) on the fully connected graph to naturally capture asynchronous spatio-temporal dependencies. Based on the spatio-temporal relationship graph convolutional operator improved from the relational graph convolution, after completing the iteration of the spatio-temporal relationship graph convolutional structure, decode the nodes to be predicted in the future and perform regression prediction. S4: Online prediction and practical application: After obtaining real-time sensor data and appending it to the time-stamp nodes of the corresponding variables in the graph, perform online measurement. The system can use the prediction results for various decision-making scenarios such as process control, anomaly warning, energy consumption optimization, or soft sensor output.

2. The prediction method for irregular time series data in process industry according to claim 1, characterized in that: In S1, the historical observation set of the nth sensor is denoted as where t n,i represents the ith observation timestamp of the sensor, and z n,i represents the corresponding observation value; K n is the total number of data points observed by the sensor during the historical period. The historical observations of all sensors can be combined and denoted as If there are obvious noises or outliers in the data, no interpolation or alignment operations are required to retain the asynchronous timing information of the original data and provide as many effective features as possible for subsequent modeling.

3. A prediction method for irregular time-series data in process industry according to claim 2, characterized in that: In S2, for the nth sensor, in addition to the historical observation set in S1, another set of future timestamps is defined for prediction. The set of all time points of this sensor can be expressed as The corresponding node features can be written as: where z unk is used as a placeholder and is learned or overwritten by the model during the prediction process; each node also carries a variable identifier for distinguishing nodes from different sensors (variables); all the above nodes (historical and future) are merged to obtain the total number of nodes and a directed edge is established for any node pair (i, j) to form a fully connected graph; Record is the original feature matrix of all nodes (where C is the original feature dimension of a single node), is the timestamp vector corresponding to all nodes, is the variable identification vector corresponding to all nodes; Define the conversion function an irregular time series and a set of future timestamps map to 4. A prediction method for irregular time series data in process industry according to claim 1, characterized in that: In S3, to reduce the computational complexity, construct a "sandwich"-type hierarchical aggregation structure, reduce the resolution of the graph through the "supernode" mechanism, and then perform local-global information fusion, which specifically includes the following steps: S301: Configure parameters and construct the structure of the spatio-temporal relationship graph convolutional neural network model. S302: Generate supernodes and perform "sandwich"-type hierarchical convolution. S303: Train the model through a decoder and a loss function to ensure that the model can effectively learn the spatio-temporal interaction rules in irregular time-series data.

5. A prediction method for irregular time series data in process industry according to claim 4, characterized in that: In S301, the parameters include initial node features, time encoding, and variable identification encoding. To highlight the weakening of the dependence of nodes with a relatively large time difference, the present invention uses normalization with a time factor during aggregation. Denote as the normalization coefficient for node j to propagate messages to node i, which can be defined as follows: where e r is a learnable scalar weight associated with a relationship of a pair of variables r (i.e., in one of the relationships), used to balance the relative influence between different variables; p i , p j are the temporal embedding vectors of nodes i and j respectively. After combining the above decoupling and normalization strategies, the update of node i at the k-th layer can be written as Among them is the hidden vector of node i at the k-th layer, is the transformation matrix for the corresponding relationship r, and σ(·) is the activation function, is the self-connection transformation matrix, and the present invention abstracts the entire layer operation as H (k+1) = STRGCNLayer(H (k) , P, S, Θ (k) ) where H (k) is the hidden vector of the k-th layer, P is the temporal encoding, and S is the spatial encoding Θ (k) contains all the learnable parameters of this layer (such as Q, φ(·), e r , W0), Q is a learnable matrix, and multiple layers stacked can gradually refine higher-order spatio-temporal correlation information.

6. The prediction method for irregular time series data in process industry according to claim 4, wherein: In S302, the original nodes are segmented or sampled according to the time axis, and a batch of super nodes are aggregated. Each super node can represent a unified abstraction within a certain time range (or several adjacent nodes), and let their spatio-temporal encodings be denoted as P respectively. β And S β , convolution is performed with super nodes as units in the middle layer to reduce the graph size.

7. A prediction method for irregular time series data in process industry according to claim 4, characterized in that: In S302, the "sandwich"-type hierarchical convolution includes the following steps: When updating the k-th layer, first perform a spatio-temporal relationship graph convolution propagation on all original nodes and supernodes at the bottom layer to obtain the supernode representation H β,(k) ; then perform convolution update on itself to obtain H' β,(k) ; finally, map the information of back to the original node resolution at the top layer to generate H' (k+1) . The three steps are recorded as follows: H β,(k) = STRGCNLayer(H (k) , P, S, Θ bottom,(k) ) H′ β,(k) = STRGCNLayer(H β,(k) , P β , S β , Θ middle,(k) ) H′ (k+1) = STRGCNLayer(H′ β,(k) , P β , S β , Θ top,(k) ) Finally, adopt a residual mechanism H (k+1) = H' (k+1) + H (k) which not only retains the underlying information but also incorporates the global context of the supernode layer. This process can be regarded as a composite layer, which can complete the multi-granularity aggregation of the entire graph with less computational cost.

8. A prediction method for irregular time series data in process industry according to claim 4, characterized in that: In S303, it includes the following steps: (1) Prediction task and decoding After completing the iteration of the spatio-temporal relational graph convolution structure, the final representation of each node is obtained For the nodes to be predicted in the future, their corresponding representations can be mapped to real values through a decoder (multi-layer perceptron MLP): Among them has the same dimension as the real observation z n,j and is used for regression prediction. (2) Loss function For the regression scenario, mean squared error (MSE) or mean absolute error (MAE) can be used. Taking MSE as an example: where is the set of all future timestamps, is the model prediction value, z n,j is the true value. To balance the model complexity, L2 regularization can be introduced: Θ contains all the trainable parameters of the model, and λ is the regularization coefficient used to prevent overfitting. (3) Training process Divide historical observations into a training set and a validation set, and iteratively minimize on the training set through Stochastic Gradient Descent (SGD) or Adaptive Optimization Algorithm (Adam). Monitor the error decline trend on the validation set. If the convergence condition or the maximum number of iterations is reached, stop the training. At this time, the model can effectively learn the spatial-temporal interaction rules in irregular time series data.

9. A prediction method for irregular time-series data in process industry according to claim 1, characterized in that: In S4, it includes the following steps: S401: Construct a new node and update the graph: For the newly arrived real-time observation (t new , z new ), according to the variable index n, add it as a new node to to obtain the updated (X, T, S) matrix. S402: Forward reasoning: Input the updated node features into the trained spatio-temporal relational graph convolutional model. Combining the convolutional calculations of each layer, the prediction results for the current or future timestamps can be obtained. S403: Output prediction values and decisions: The system can use the prediction results for various decision-making scenarios such as process control, anomaly warning, energy consumption optimization, or soft sensor output. Since the data in the present invention is not interpolated or pre-aligned, the model can still ensure efficient update and inference when new data arrives.

10. A prediction method for irregular time-series data in process industry according to any one of claims 1-9, characterized in that: The sensors are used to monitor key process variables including but not limited to temperature, flow rate, and pressure.

Images