A prediction method for irregular time series data in process industry

Abstract

The application provides a prediction method for irregular time series data in a process industry, and belongs to the technical field of industrial time series data prediction, and comprises the following steps: data acquisition and preprocessing; constructing a graph representation of irregular time series data; constructing and training a space-time relationship graph convolutional neural network model; online prediction, and practical application, the space-time relationship graph convolutional neural network model comprises: a full connection graph conversion module, a space-time relationship graph convolutional layer and a hierarchical sandwich structure, the application constructs a full connection graph representation without pre-alignment for irregularly sampled multivariate time series data in a process industry environment, and high-precision prediction of an asynchronous and complex space-time dependence structure is realized based on a space-time relationship graph convolutional module, and the hierarchical sandwich strategy further guarantees the calculation efficiency and model performance on large-scale graph data, and is suitable for real-time prediction and fault early warning and other requirements of an industrial site containing a large number of asynchronous sensor measuring points.

Description

Technical Field

[0001] This invention belongs to the field of industrial time series data prediction technology, specifically relating to a prediction method for irregular time series data in process industries. Background Technology

[0002] For prediction methods of irregular time series data in process industries, it is necessary to combine the dynamic characteristics of time series data with the special needs of industrial scenarios (such as high noise, multivariate coupling, physical constraints, etc.). The data has the following characteristics: (1) Irregularity: the data sampling interval is not fixed (such as event-triggered acquisition, sensor failure, manual recording, etc.); (2) Multivariate coupling: there are strong physical / chemical correlations between variables in process industries (such as temperature, pressure, flow, etc.); (3) Noise and missing data: sensor noise, data loss, and outliers are common; (4) Long-term and short-term dependence: it is necessary to capture both short-term fluctuations (such as equipment status) and long-term trends (such as process degradation).

[0003] Time series forecasting technology has broad application potential in process industries (such as chemical and oil refining industries). Predicting future sequences based on historical data can not only enable early monitoring and warning of processes, but is also widely used in soft sensors and digital twin systems.

[0004] Irregular time series data refers to time series with variable sampling intervals and asynchronous sampling points between variables, such as... Figure 1 As shown, Figure 1 (a) is a regular time series with a fixed sampling interval and sampling points of different variables v in time pairs. Figure 1 (b) illustrates the characteristics of irregular time series, a type of data prevalent in industries such as manufacturing, healthcare, and finance. Variables exhibit irregular sampling intervals, with different variables possessing varying sampling frequencies and misaligned timestamps. Compared to regular time series, irregular time series demonstrate more complex and variable spatiotemporal structures. However, most current time series prediction methods are based on the assumption of regular sampling. For example, the multidimensional spatiotemporal attention method proposed in patent CN115034325B and the pre-training method for multivariate time series in the process industry proposed in patent CN117312851A do not consider the problem of irregular sampling.

[0005] In recent years, several methods for processing irregular time series have emerged. For example, patent CN115329166A proposes a classification method suitable for irregular time series data, but this method focuses on classification tasks and does not address the prediction problem that this invention addresses. Furthermore, patents CN118194242A and CN118245474B both propose specialized methods for predicting irregular time series data, but they both employ a data pre-alignment strategy, i.e., expanding univariate observations to cover all observed timestamps. This strategy causes the size of the input data matrix to increase exponentially with the number of variables, resulting in a significant increase in memory usage and computational costs during the training and inference phases.

[0006] In contrast, the method proposed in this invention no longer relies on data pre-alignment, but instead models irregular time series as a fully connected graph structure, where nodes represent actual observation points and edges represent potential spatiotemporal relationships between observation points. This graph representation avoids the problem of the input matrix size increasing exponentially with the number of variables. Furthermore, through the representation learning mechanism of graph neural networks, this method can naturally and efficiently handle different sampling frequencies and unaligned timestamps among variables, directly capturing the complex asynchronous spatiotemporal dependencies in the data, significantly reducing computational complexity while improving the predictive performance for irregular industrial time series data. Summary of the Invention

[0007] To address the shortcomings of existing technologies, this invention proposes a prediction method for irregular time-series data in the process industry. In the process industry environment, a fully connected graph representation without pre-alignment is constructed for irregularly sampled multivariate time-series data, and high-precision prediction of asynchronous and complex spatiotemporal dependency structures is achieved based on the spatiotemporal relationship graph convolution module.

[0008] To solve the above problems, the present invention adopts the following technical solution:

[0009] A prediction method for irregular time-series data in process industries, the method comprising the following steps:

[0010] S1: Data Acquisition and Preprocessing: N sensors are deployed in the required scenario, where n represents the nth sensor, to monitor different process parameters. The acquired data is preprocessed, and extreme anomalies or obvious measurement errors are eliminated or corrected.

[0011] S2: Constructing a graph representation of irregular time series data: Treat all historical observation points and future time points that need to be predicted as nodes in a fully connected graph, thereby using graph neural networks to flexibly model the relationships between irregular time series;

[0012] S3: Constructing and training a spatiotemporal graph convolutional neural network model: Introducing a spatiotemporal graph convolutional network (STRGCN) on a fully connected graph to naturally capture asynchronous spatiotemporal dependencies. Based on the spatiotemporal graph convolution operator improved by graph convolution, after completing the iteration of the spatiotemporal graph convolution structure, the future nodes to be predicted are decoded and regression predictions are performed.

[0013] S4: Online prediction and practical application: After acquiring real-time sensor data and appending it to the timestamp nodes of the corresponding variables in the graph, the system can perform online measurement. The prediction results can be used for various decision-making scenarios such as process control, anomaly warning, energy consumption optimization, or soft sensor output.

[0014] Furthermore, in S1, let the historical observation set of the nth sensor be denoted as . ,in This represents the l-th observation timestamp of the sensor. This represents the corresponding observed value; This represents the total number of data points observed by the sensor during the historical period. All historical observations from all sensors can be combined and recorded as follows: If the data contains obvious noise or outliers, no interpolation or alignment operations are required, so as to preserve the asynchronous time-series information of the original data and provide as many effective features as possible for subsequent modeling.

[0015] Furthermore, in S2, for the nth sensor, in addition to the historical observation set in S1, another set of future timestamps is defined. For prediction, the set of all time points of the sensor can be represented as:

[0016] ;

[0017] The corresponding node features can be written as:

[0018] ;

[0019] in As placeholders, they are learned or overridden by the model during the prediction process; each node also carries a variable identifier. This is used to distinguish nodes from different sensors (variables); all the above nodes (historical and future) are merged to obtain the total number of nodes. And for any pair of nodes (i,j), a directed edge is established to form a fully connected graph;

[0020] remember This is the original feature matrix of all nodes (where C is the original feature dimension of a single node). This is a timestamp vector corresponding to all nodes. A variable identifier vector for all nodes;

[0021] Define conversion function irregular time series With future timestamp sets Mapped to .

[0022] Furthermore, in S3, to reduce computational load, a "sandwich" hierarchical aggregation structure is constructed. The resolution of the graph is reduced through a "supernode" mechanism, followed by local-global information fusion. Specifically, this includes the following steps:

[0023] S301: Configure parameters to construct the spatiotemporal graph convolutional neural network model structure;

[0024] S302: Generate supernodes and perform "sandwich" style hierarchical convolution;

[0025] S303: Model training is performed using a decoder and loss function to ensure that the model can effectively learn the spatial-temporal interaction patterns in irregular time-series data.

[0026] Furthermore, in S301, the parameters include initial node features, time encoding, and variable identifier encoding. To highlight the reduced dependency of node pairs with large time intervals, this invention employs normalization with a time factor during aggregation. The normalization coefficient for propagating the message from node j to node i can be defined as follows:

[0027] ;

[0028] in This represents the set of neighboring nodes of node i under relation r, that is, all nodes connected to node i through relation type r; e r These are learnable scalar weights associated with the variable r, used to balance the relative influences among different variables, where r is... A certain relationship in, This represents a set of spatial relationships, used to describe the types of relationships between different sensor variables; p i p j These are the temporal embedding vectors of nodes i and j, respectively. After combining the decoupling and normalization strategies described above, the update of node i at layer k can be written as...

[0029] ;

[0030] in This is the time transformation matrix related to relation r in the k-th layer, used to capture feature transformations in the time dimension; This is the spatial transformation matrix related to relation r in the k-th layer, used to capture feature transformations in the spatial dimension; the two are multiplied together. Achieve decoupling transformation of spatiotemporal features; It is the latent vector of node i in the k-th layer. Let r be the transformation matrix of the correspondence. For activation function, As a self-connected transformation matrix, this invention abstracts the entire layer operation as...

[0031] ;

[0032] in Let L be the latent vector matrix of all nodes in the k-th layer, where L is the total number of nodes and d is the dimension of the latent vectors. P For time encoding, S Encoding for space It contains all the learnable parameters of this layer, and stacking multiple layers can gradually extract higher-order spatiotemporal correlation information;

[0033] Furthermore, in S302, the original nodes are segmented or sampled according to the time axis, and a batch of supernodes are obtained by aggregation. Each supernode can represent a unified abstraction within a certain time range (or several adjacent nodes), and its spatiotemporal encoding is denoted as follows: and Convolution is performed in the intermediate layer on a per-supernode basis to reduce the graph size.

[0034] Furthermore, in S302, the "sandwich" layered convolution includes the following steps:

[0035] When updating the k-th level, first update all original nodes and super nodes at the bottom level. Perform a spatiotemporal graph convolutional propagation to obtain the supernode representation. Then in the intermediate layer The result is obtained by performing convolution updates. Finally, on the top floor... The information is backmapped to the original node resolution to generate Remember the three steps as follows:

[0036] ;

[0037] Finally, the residual mechanism is adopted.

[0038] ;

[0039] By preserving the underlying information and incorporating the global context of the supernode layer, this process can be viewed as a composite layer that can complete multi-granularity aggregation of the entire graph with less computational cost.

[0040] Furthermore, S303 includes the following steps:

[0041] (1) Prediction task and decoding

[0042] After iterating through the spatiotemporal graph convolutional structure, the final representation of each node is obtained. For future nodes to be predicted, their corresponding representations can be mapped to real values ​​using a decoder (Multilayer Perceptron, MLP).

[0043] ;

[0044] in Compared with actual observation They have the same dimensions and are used for regression prediction.

[0045] (2) Loss function

[0046] For regression scenarios, mean squared error (MSE) or mean absolute error (MAE) can be used. Taking MSE as an example:

[0047] ;

[0048] Where n represents the sensor index, corresponding to the nth sensor defined in S1, and l represents the index of the future timestamp to be predicted under sensor n. For the total number of future timestamps, These are the model's predicted values. This represents the true value. To balance model complexity, L2 regularization can be introduced:

[0049] ;

[0050] It contains all trainable parameters of the model. This is the regularization coefficient, used to prevent overfitting.

[0051] (3) Training process

[0052] Historical observation The dataset is divided into training and validation sets. Stochastic gradient descent (SGD) or an adaptive optimization algorithm (Adam) is used to iteratively minimize the set on the training set. The error rate is monitored on the validation set, and training stops when the convergence condition or the maximum number of iterations is reached. At this point, the model can effectively learn the spatial-temporal interaction patterns in irregular time-series data.

[0053] Furthermore, S4 includes the following steps:

[0054] S401: Construct new nodes and update the graph: for newly arrived real-time observations It is treated as a new node and added based on the variable index n. In this process, the updated (X,T,S) matrix is ​​obtained.

[0055] S402: Forward Inference: The updated node features are input into the pre-trained spatiotemporal graph convolutional model. By combining the convolutional calculations of each layer, a prediction result for the current or future timestamp can be obtained. .

[0056] S403: Output Predicted Values ​​and Decisions: The system can use the prediction results for various decision-making scenarios such as process control, anomaly early warning, energy consumption optimization, or soft sensor output. Because this invention does not interpolate or pre-align the data, the model can still ensure efficient updating and inference when new data arrives.

[0057] Furthermore, the sensor is used to monitor key process variables, including but not limited to temperature, flow rate, and pressure.

[0058] Advantages of this invention:

[0059] 1. In the process industry environment, this invention constructs a fully connected graph representation for irregularly sampled multivariate time series data without pre-alignment, and realizes high-precision prediction of asynchronous and complex spatiotemporal dependency structures based on the spatiotemporal graph convolution module.

[0060] 2. The layered "sandwich" strategy further ensures computational efficiency and model performance on large-scale graph data, and is suitable for real-time prediction and fault early warning needs in industrial sites with a large number of asynchronous sensor measurement points. Attached Figure Description

[0061] To more clearly illustrate the specific embodiments of the present invention, the accompanying drawings used in the description of the specific embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0062] Figure 1 This is a comparison chart of regular and irregular time series in the background technology.

[0063] Figure 2 This is a flowchart of the method;

[0064] Figure 3 A schematic diagram illustrating the spatiotemporal graph convolutional neural network model proposed in this method;

[0065] Figure 4 A schematic diagram of supernode generation and hierarchical sandwich structure. Detailed Implementation

[0066] The specific embodiments of the present invention are described below to enable those skilled in the art to understand the present invention. However, it should be understood that the present invention is not limited to the scope of the specific embodiments. For those skilled in the art, the main variations are obvious within the spirit and scope of the present invention as defined and determined by the appended claims. All inventions utilizing the concept of the present invention are protected.

[0067] It should be noted that the technical terms used in this invention are only for the purpose of describing specific embodiments and are not intended to limit the scope of protection of this invention. Unless otherwise specified, all instruments and equipment used in the following embodiments of this invention can be purchased from the market or prepared by existing methods.

[0068] like Figure 3 As shown, the present invention constructs an overview of the proposed spatiotemporal graph convolutional neural network model, including the following key components: (a) a fully connected graph transformation module, which converts irregular time series data into a compactly represented fully connected graph; (b) a spatiotemporal graph convolutional layer, which aims to capture asynchronous spatiotemporal dependencies; and (c) a hierarchical sandwich structure, which integrates local and global semantic relationships while alleviating computational complexity.

[0069] Example 1

[0070] like Figure 2 This solution provides a prediction method for irregular time-series data in the process industry, specifically including the following steps:

[0071] S1: Data Acquisition and Preprocessing.

[0072] N sensors are deployed in the required scenario to monitor different process parameters. The collected data is preprocessed, and extreme anomalies or obvious measurement errors are eliminated or corrected.

[0073] In process industries (such as chemical and oil refining), multiple sensors are typically deployed to monitor key process variables (such as temperature, pressure, and flow rate). Due to differences in the hardware characteristics or sampling strategies of each sensor, the obtained time-series data often exhibits characteristics such as: inconsistent sampling intervals (unequal sampling time intervals for the same variable) and asynchronous sampling between variables (the observation timestamps of different variables are not aligned, which may result in significant time deviations). Therefore, N sensors (which can also be denoted as variables v1, ..., vN) are typically deployed to monitor different process parameters, such as temperature, flow rate, and pressure. Due to differences in the hardware characteristics or sampling strategies of each sensor, their sampling intervals are not fixed, and the observation timestamps between different sensors are often not aligned.

[0074] Let the historical observation set of the nth sensor be . ,in This represents the l-th observation timestamp of the sensor. This represents the corresponding observed value; This represents the total number of data points observed by the sensor during the historical period. All historical observations from all sensors can be combined and recorded as follows: If the data contains obvious noise or outliers, no interpolation or alignment operations are required, so as to preserve the asynchronous time-series information of the original data and provide as many effective features as possible for subsequent modeling.

[0075] S2: Construct a graph representation of irregular time-series data.

[0076] By treating all historical observation points and future time points that need to be predicted as nodes in a fully connected graph, graph neural networks can be used to flexibly model the relationships between irregular time series.

[0077] For the nth sensor, in addition to the historical observation set in S1, another set of future timestamps is defined. For prediction, the set of all time points of the sensor can be represented as:

[0078] ;

[0079] The corresponding node features can be written as:

[0080] ;

[0081] in As placeholders, they are learned or overridden by the model during the prediction process; each node also carries a variable identifier. This is used to distinguish nodes from different sensors (variables); all the above nodes (historical and future) are merged to obtain the total number of nodes. And for any pair of nodes (i,j), a directed edge is established to form a fully connected graph;

[0082] remember This is the original feature matrix of all nodes (where C is the original feature dimension of a single node). This is a timestamp vector corresponding to all nodes. A variable identifier vector for all nodes;

[0083] Define conversion function irregular time series With future timestamp sets Mapped to .

[0084] In this way, even if different sensors have different sampling frequencies and time distributions, they can be processed uniformly under the same graph structure.

[0085] S3: Constructing and training a spatiotemporal graph convolutional neural network model.

[0086] A spatiotemporal relational graph convolutional network (STRGCN) is introduced on a fully connected graph to naturally capture asynchronous spatiotemporal dependencies. Based on the spatiotemporal relational graph convolution operator improved by relational graph convolution, after completing the iteration of the spatiotemporal relational graph convolution structure, the future nodes to be predicted are decoded and regression prediction is performed.

[0087] Optional, such as Figure 4 As shown, a hierarchical "sandwich" structure can be used to effectively reduce computational overhead on large-scale fully connected graphs. Figure 4 (a) Shows how supernodes are generated for each variable by uniformly sampling nodes along the time axis, with a predefined window length. Figure 4 (b) provides an intuitive understanding of the bottom, middle and top layers in a sandwich structure.

[0088] Specifically, the following steps are included:

[0089] S301: Configure parameters to construct the spatiotemporal relationship graph convolutional neural network model structure.

[0090] The parameters include initial node features, time encoding, and variable identifier encoding. To highlight the reduced dependency of node pairs with large time intervals, this invention employs normalization with a time factor during aggregation. The normalization coefficient for propagating the message from node j to node i can be defined as follows:

[0091] ;

[0092] in This represents the set of neighboring nodes of node i under relation r, that is, all nodes connected to node i through relation type r; e r These are learnable scalar weights associated with the variable r, used to balance the relative influences among different variables, where r is... A certain relationship in, This represents a set of spatial relationships, used to describe the types of relationships between different sensor variables; p i p j These are the temporal embedding vectors of nodes i and j, respectively. After combining the decoupling and normalization strategies described above, the update of node i at layer k can be written as...

[0093] ;

[0094] in This is the time transformation matrix related to relation r in the k-th layer, used to capture feature transformations in the time dimension; This is the spatial transformation matrix related to relation r in the k-th layer, used to capture feature transformations in the spatial dimension; the two are multiplied together. Achieve decoupling transformation of spatiotemporal features; It is the latent vector of node i in the k-th layer. Let r be the transformation matrix of the correspondence. For activation function, As a self-connected transformation matrix, this invention abstracts the entire layer operation as...

[0095] ;

[0096] in Let L be the latent vector matrix of all nodes in the k-th layer, where L is the total number of nodes and d is the dimension of the latent vectors. P For time encoding, S Encoding for space It contains all the learnable parameters of the layer, and stacking multiple layers can gradually extract higher-order spatiotemporal correlation information.

[0097] S302: Generate supernodes and perform "sandwich" layered convolution.

[0098] (1) Supernode generation: such as Figure 3 As shown in (a), the original nodes are segmented or sampled according to the time axis, and a batch of supernodes are obtained by aggregation. Each supernode can represent a unified abstraction within a certain time range (or several adjacent nodes), and its spatiotemporal encoding is denoted as follows: and Convolution is performed on the supernodes in the intermediate layers to reduce the graph size.

[0099] (2) Sandwich-style layered convolution:

[0100] like Figure 3 As shown in (b), when updating the k-th layer, first update all original nodes and super nodes at the bottom layer. Perform a spatiotemporal graph convolutional propagation to obtain the supernode representation. Then in the intermediate layer The result is obtained by performing convolution updates. Finally, on the top floor... The information is backmapped to the original node resolution to generate Remember the three steps as follows:

[0101] ;

[0102] Finally, the residual mechanism is adopted.

[0103] ;

[0104] This process, which preserves underlying information while incorporating the global context of the supernode layer, can be viewed as a composite layer that enables multi-granularity aggregation of the entire graph with relatively low computational cost.

[0105] S303: Model training is performed using a decoder and loss function to ensure that the model can effectively learn the spatial-temporal interaction patterns in irregular time-series data.

[0106] (1) Prediction task and decoding

[0107] After iterating through the spatiotemporal graph convolutional structure, the final representation of each node is obtained. For future nodes to be predicted, their corresponding representations can be mapped to real values ​​using a decoder (Multilayer Perceptron, MLP).

[0108] ;

[0109] in Compared with actual observation They have the same dimensions and are used for regression prediction.

[0110] (2) Loss function

[0111] For regression scenarios, mean squared error (MSE) or mean absolute error (MAE) can be used. Taking MSE as an example:

[0112] ;

[0113] Where n represents the sensor index, corresponding to the nth sensor defined in S1, and l represents the index of the future timestamp to be predicted under sensor n. For the total number of future timestamps, These are the model's predicted values. This represents the true value. To balance model complexity, L2 regularization can be introduced:

[0114] ;

[0115] It contains all trainable parameters of the model. This is the regularization coefficient, used to prevent overfitting.

[0116] (3) Training process

[0117] Historical observation The dataset is divided into training and validation sets. Stochastic gradient descent (SGD) or an adaptive optimization algorithm (Adam) is used to iteratively minimize the set on the training set. The error rate is monitored on the validation set, and training stops when the convergence condition or the maximum number of iterations is reached. At this point, the model can effectively learn the spatial-temporal interaction patterns in irregular time-series data.

[0118] S4: Online prediction and practical application.

[0119] After acquiring real-time sensor data and appending it to the timestamp nodes of the corresponding variables in the graph, the system performs online measurements. The prediction results can be used for various decision-making scenarios such as process control, anomaly warning, energy consumption optimization, or soft sensor output.

[0120] Specifically, the steps include the following:

[0121] S401: Construct new nodes and update the graph: for newly arrived real-time observations It is treated as a new node and added based on the variable index n. In this process, the updated (X,T,S) matrix is ​​obtained.

[0122] S402: Forward Inference: The updated node features are input into the pre-trained spatiotemporal graph convolutional model. By combining the convolutional calculations of each layer, a prediction result for the current or future timestamp can be obtained. .

[0123] S403: Output Predicted Values ​​and Decisions: The system can use the prediction results for various decision-making scenarios such as process control, anomaly early warning, energy consumption optimization, or soft sensor output. Because this invention does not interpolate or pre-align the data, the model can still ensure efficient updating and inference when new data arrives.

[0124] Through the above steps, this invention constructs a fully connected graph representation without pre-alignment for irregularly sampled multivariate time-series data in a process industry environment, and achieves high-precision prediction of asynchronous and complex spatiotemporal dependency structures based on a spatiotemporal graph convolution module. The hierarchical "sandwich" strategy further ensures computational efficiency and model performance on large-scale graph data, making it suitable for real-time prediction and fault early warning needs in industrial settings with a large number of asynchronous sensor measurement points.

[0125] Example 2

[0126] As a preferred embodiment of the present invention, the following embodiments aim to illustrate the specific implementation and experimental evaluation results of the present invention in the task of predicting irregular multivariate time series data, so as to help those skilled in the art to more deeply understand and apply the technical solution. It should be emphasized that the data sources, model parameters, and index results given in this embodiment are not the only limitations of the present invention, and related details can be flexibly adjusted according to actual needs, specifically including the following steps:

[0127] 1. Dataset and Experiment Setup.

[0128] (1) Data source

[0129] This embodiment is validated on a representative real-world process industry irregular multivariate time series dataset, DIST-Irr. DIST-Irr is collected from sensor data of a dual-tower distillation process in a chemical plant in China. The DIST-Irr dataset originates from the methanol distillation process in the chemical plant and covers distributed control system data from July 1 to August 1, 2023. The dataset contains 37 sensors (variables) and consists of 133,920 data points, ensuring high-resolution data collection.

[0130] For each of the above datasets, it is randomly divided into a training set, a validation set, and a test set in a ratio of 60%:20%:20%. Let the dataset contain 37 sensors (variables), where the historical observation sets of each variable are... The set of timestamps that need to be predicted in the future .

[0131] (2) Experimental environment and setup

[0132] Training and inference were performed on a GPU server equipped with an NVIDIA GeForce RTX 4090 (24GB) of video memory; the operating system was Linux, and the deep learning framework used was PyTorch.

[0133] For the DIST-irr dataset: use observations from the past hour (multivariate irregular time series) to predict the values ​​of each variable for the next hour.

[0134] 2. Specific parameter configuration

[0135] Consistent with the notation in Example 1, the node features, dimensions of time encoding and variable encoding, and key hyperparameters in the hierarchical "sandwich" structure are all given here in this example.

[0136] (1) Initial features and temporal coding dimension

[0137] Let the original node feature matrix be... The input dimension C varies depending on the dataset and is obtained through linear mapping.

[0138]

[0139] In this embodiment, D=64.

[0140] timestamp vector Through continuous-time encoding function get The time encoding method can use sine and cosine basis functions or learnable embeddings; this embodiment uses a hybrid sine and cosine method.

[0141] (2) Construction and training of spatiotemporal graph convolutional networks

[0142] The low-rank decomposition matrix Q corresponds to: Learnable matrices in This is used to capture the time difference between nodes i and j.

[0143] Variable Embedding Function : Identifier for each sensor variable Mapped to a 64-dimensional vector Then, it is used together with temporal embeddings for computation. .

[0144] Normalization coefficient: Let

[0145]

[0146] Where e r The scalar weights of a variable with respect to r are all learned during training.

[0147] (3) Layered "sandwich" structure

[0148] Hierarchical supernodes: The original nodes are aggregated into several "supernodes" according to the time sequence. In this embodiment, the number of supernodes varies depending on the dataset, so as to build a smaller graph in the intermediate layer and reduce the computational overhead.

[0149] Three-level update: First, at the bottom level, update all original nodes and supernodes. Perform a spatiotemporal graph convolutional propagation to obtain the supernode representation. Then in the intermediate layer The result is obtained by performing convolution updates. Finally, on the top floor... The information is backmapped to the original node resolution to generate The three steps are as follows:

[0150]

[0151] Finally, the residual mechanism is adopted.

[0152]

[0153] By preserving the underlying information and incorporating the global context of the supernode layer, this process can be viewed as a composite layer that can complete multi-granularity aggregation of the entire graph with less computational cost.

[0154] (4) Training loss and hyperparameters

[0155] The loss function uses the root mean square error (MSE).

[0156]

[0157] in, , middle Take 10 -3 .

[0158] The optimizer used is Adam, with an initial learning rate of 1×10⁻⁶. −3 Set the batch size to 32.

[0159] The maximum number of training epochs is 30, and observations are made on the validation set. The wheel stops descending prematurely.

[0160] 3. Experimental Results and Beneficial Effects

[0161] Baseline method: The state-of-the-art model t-PatchGNN in the field of irregular time series analysis was selected as the baseline model for comparison. Some key components were removed from the Spatiotemporal Relationship Graph Convolution (STRGCN) model proposed in this patent to verify the effectiveness of the proposed scheme, including re-GCN (replacing the proposed spatiotemporal relationship graph convolution operator with the traditional graph convolution operator) and w / o-Sandwich (removing the hierarchical sandwich structure).

[0162] Table 1 shows the experimental results of the model on the DIST-Irr multivariate time series prediction dataset (lower MSE is better):

[0163] Table 1: Experimental Results of Irregular Multivariate Temporal Prediction Task

[0164]

[0165] Experimental results show that the proposed spatiotemporal graph convolutional model STRGCN achieves the best prediction performance. Furthermore, compared to t-PatchGNN, which employs a patch-based temporal segmentation strategy, STRGCN can capture asynchronous temporal relationships at both the node level and local fine-grained levels, thus achieving an approximately 12% reduction in MSE. Removing the key innovations of this patent—the "spatiotemporal graph convolutional operator" and the "layered sandwich structure"—results in a decrease in prediction performance, demonstrating the beneficial effects of the proposed content.

[0166] The above examples have provided a detailed description of this embodiment, but the content is only a preferred embodiment and should not be considered as limiting the scope of implementation of this embodiment; all equivalent changes and improvements made in accordance with the scope of this embodiment should still fall within the patent coverage of this embodiment.

Claims

1. A prediction method for irregular time-series data in process industries, characterized in that: The method includes the following steps: S1: Data Acquisition and Preprocessing: N sensors are deployed in the required scenario, where n represents the nth sensor, to monitor different process parameters. The acquired data is preprocessed, and extreme anomalies or obvious measurement errors are eliminated or corrected. S2: Constructing a graph representation of irregular time series data: Treat all historical observation points and future time points that need to be predicted as nodes in a fully connected graph, thereby using graph neural networks to flexibly model the relationships between irregular time series; S3: Constructing and Training a Spatiotemporal Relationship Graph Convolutional Neural Network Model: A spatiotemporal relationship graph convolutional network is introduced onto a fully connected graph to naturally capture asynchronous spatiotemporal dependencies. Based on an improved spatiotemporal relationship graph convolution operator, after iterating through the spatiotemporal relationship graph convolutional structure, future nodes to be predicted are decoded and regression predictions are performed. To reduce computational cost, a "sandwich" hierarchical aggregation structure is constructed. The resolution of the graph is reduced through a "supernode" mechanism, and then local-global information fusion is performed. The specific steps include the following: S301: Configure parameters to construct the spatiotemporal graph convolutional neural network model structure; The parameters include initial node features, time encoding, and variable identifier encoding. To highlight the reduced dependency of node pairs with large time intervals, normalization with a time factor is used during aggregation. Let be the normalization coefficient for propagating the message from node j to node i, defined as follows: ； in This represents the set of neighboring nodes of node i under relation r, that is, all nodes connected to node i through relation type r; e r These are learnable scalar weights associated with the variable r, used to balance the relative influences among different variables, where r is... A certain relationship in, This represents a set of spatial relationships, used to describe the types of relationships between different sensor variables; p i p j These are the temporal embedding vectors of nodes i and j, respectively. After applying the normalization strategy described above, the update of node i at layer k is written as... ； in This is the time transformation matrix related to relation r in the k-th layer, used to capture feature transformations in the time dimension; This is the spatial transformation matrix related to relation r in the k-th layer, used to capture feature transformations in the spatial dimension; the two are multiplied together. Achieve decoupling transformation of spatiotemporal features; It is the latent vector of node i in the k-th layer. Let r be the transformation matrix of the correspondence. For activation function, For the self-connected transformation matrix, the entire layer of operations is abstracted as ； in Let L be the latent vector matrix of all nodes in the k-th layer, where L is the total number of nodes and d is the dimension of the latent vectors. P For time encoding, S Encoding for space It contains all the learnable parameters of this layer, and stacking multiple layers can gradually extract higher-order spatiotemporal correlation information; S302: Generate supernodes, perform "sandwich" layered convolution, segment or sample the original nodes according to the time axis, and aggregate to obtain a batch of supernodes. Each supernode represents a unified abstraction within a certain time range, and its spatiotemporal encoding is denoted as follows: and The "sandwich" layered convolution, which performs convolution on a per-supernode basis in the intermediate layer to reduce the graph size, includes the following steps: When updating the k-th level, first update all original nodes and super nodes at the bottom level. Perform a spatiotemporal graph convolutional propagation to obtain the supernode representation. Then in the intermediate layer The result is obtained by performing convolution updates. Finally, on the top floor... The information is backmapped to the original node resolution to generate Remember the three steps as follows: ； Finally, the residual mechanism is adopted. ； It retains the underlying information while incorporating the global context of the supernode layer. This process is regarded as a composite layer, which can complete the multi-granularity aggregation of the entire graph with less computational cost. S303: Model training is performed using a decoder and loss function to ensure that the model effectively learns the space-time interaction patterns in irregular temporal data, including the following steps: (1) Prediction task and decoding After iterating through the spatiotemporal graph convolutional structure, the final representation of each node is obtained. For future nodes to be predicted, their corresponding representations can be mapped to real values ​​through the decoder: ； in Compared with actual observation They have the same dimensions and are used for regression prediction; (2) Loss function For regression scenarios, mean squared error or mean absolute error can be used: ； Where n represents the sensor index, corresponding to the nth sensor defined in S1, and l represents the index of the future timestamp to be predicted under sensor n. For the total number of future timestamps, These are the model's predicted values. The actual value is used; to balance model complexity, L2 regularization is introduced: ； It contains all trainable parameters of the model. This is the regularization coefficient, used to prevent overfitting; (3) Training process Historical observation The dataset is divided into training and validation sets. The minimum value is iteratively minimized on the training set using stochastic gradient descent or adaptive optimization algorithms. The error decrease trend is monitored on the validation set. Training is stopped when the convergence condition or the maximum number of iterations is reached. At this time, the model can effectively learn the space-time interaction pattern in irregular time series data. S4: Online prediction and practical application: After acquiring real-time sensor data and appending it to the timestamp nodes of the corresponding variables in the spatiotemporal relationship graph, online prediction is performed. The system can use the prediction results for various decision-making scenarios such as process control, anomaly warning, energy consumption optimization, or soft sensor output.

2. The prediction method for irregular time-series data in process industries according to claim 1, characterized in that: In S1, let the historical observation set of the nth sensor be denoted as . ,in This represents the l-th observation timestamp of the sensor. This represents the corresponding observed value; This represents the total number of data points observed by the sensor during the historical period. All historical observations from all sensors can be combined and recorded as follows: If the data contains obvious noise or outliers, no interpolation or alignment operations are required, so as to preserve the asynchronous time-series information of the original data and provide as many effective features as possible for subsequent modeling.

3. The prediction method for irregular time-series data in process industries according to claim 2, characterized in that: In S2, for the nth sensor, in addition to the historical observation set in S1, another set of future timestamps is defined. Used for prediction, where M n This represents the number of future time points that the nth sensor needs to predict. Let m be the future time to be predicted for the nth sensor. The set of all time points of this sensor is represented as: ； Where K n This represents the total number of historical observation points for the nth sensor. This indicates the Kth sensor. n A historical observation moment; This indicates that the Mth sensor n For each future time to be predicted, the corresponding node features are written as: ； in As placeholders, they are learned or overridden by the model during the prediction process; each node also carries a variable identifier. This is used to distinguish nodes from different sensors; all the above nodes are then merged to obtain the total number of nodes. And for any pair of nodes (i,j), a directed edge is established to form a fully connected graph; remember Let C be the original feature matrix of all nodes, where C is the original feature dimension of a single node. This is a timestamp vector corresponding to all nodes. A variable identifier vector for all nodes; Define conversion function irregular time series With future timestamp sets Mapped to .

4. The prediction method for irregular time-series data in process industries according to claim 1, characterized in that: S4 includes the following steps: S401: Construct new nodes and update the graph: for newly arrived real-time observations Based on the variable index n, it is treated as a new node and added to the fully connected graph to obtain the updated (X,T,S) matrix; S402: Forward Inference: The updated node features are input into the pre-trained spatiotemporal graph convolutional model. By combining the convolutional calculations of each layer, a prediction result for the current or future timestamp can be obtained. ; S403: Output Predictions and Decisions: The system can use the prediction results for various decision-making scenarios such as process control, anomaly warning, energy consumption optimization, or soft sensor output. Since the data is not interpolated or pre-aligned, the model can still ensure efficient updating and inference when new data arrives.

5. A prediction method for irregular time-series data in process industries according to any one of claims 1-4, characterized in that: The sensor is used to monitor key process variables, including temperature, flow rate, and pressure.

Images