A distributed parameter system state prediction method based on multi-source spatiotemporal information

Through the multi-source spatiotemporal network model and residual convolution LSTM network, the timing and spatiotemporal features are extracted and fused, the multi-source feature ignorance problem in complex nonlinear parameter system modeling is solved, the accuracy and generalization ability of the model are improved, and the accuracy and generalization ability of the model are realized. Effective prediction of the status of the parameter system.

CN114330143BActive Publication Date: 2024-11-01QINGDAO UNIV OF TECH
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202210078924.8
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2022-01-24
Publication Date
2024-11-01
Estimated Expiration
2042-01-24

AI Technical Summary

Technical Problem

The existing parameter system state prediction methods are difficult to effectively model complex nonlinear distributed parameter systems, especially ignoring the multi-source spatiotemporal characteristics, resulting in insufficient model accuracy and generalization capabilities.

Method used

A multi-source spatiotemporal network model is adopted to build a model that can extract timing and spatiotemporal features through data conversion, timing feature extraction, spatiotemporal feature extraction and feature fusion through residual convolutional LSTM network, and state prediction is carried out using ConvLSTM.

Benefits of technology

The fitting accuracy and generalization ability of the model are improved, and the state of complex parameter systems can be effectively predicted, especially in large-scale industrial processes, which reduces the risk of gradient vanishing and gradient explosion and enhances the stability of the model.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN114330143B_ABST
    Figure CN114330143B_ABST
Patent Text Reader

Abstract

Aiming at the problem that the process state of nonlinear distributed parameter systems is difficult to predict, the present invention proposes a new multi-source spatiotemporal modeling method to improve the real-time measurement accuracy. In order to solve the modeling problem of distributed parameter systems with multi-source spatiotemporal characteristics, a multi-source spatiotemporal network model is constructed. The model is based on the long short-term memory (LSTM) network to extract process variable information with time series characteristics, and based on the convolutional long short-term memory (ConvLSTM) network to extract spatiotemporal variable information with spatiotemporal characteristics. Comprehensive use of various process information is conducive to improving the fitting accuracy of the model, and can also improve the generalization ability of the model. In order to prevent the gradient from disappearing or exploding due to the increase in the number of network layers when extracting complex spatiotemporal data features, a residual network model structure based on ConvLSTM is proposed for training the model. Since the residual network can avoid model degradation, the established model can fully extract the spatiotemporal characteristics of the process.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to a parameter system state prediction method, in particular to a distributed parameter system state prediction method based on multi-source spatiotemporal information. Background Art

[0002] Numerous physical and chemical industrial processes (such as thermal and convection-diffusion-reaction processes) are nonlinear distributed parameter systems (DPS) because their inputs, outputs, and states can change in time and space. As industry develops towards a smart industrial model, safer production and fewer failures are particularly important for industrial processes, especially for DPS with spatiotemporal characteristics. In order to ensure safe industrial production, accurate prediction and early detection of process states are necessary. In order to perform system simulation and process dynamic prediction, a suitable system model is usually required to describe the process.

[0003] A lot of research has been done to describe DPS.Depending on whether the DPS is known or not, DPS modeling can be roughly divided into two categories, namely known DPS modeling and unknown DPS modeling.

[0004] For known DPS, partial differential equations (PDEs) derived from first principles knowledge are usually used to describe the system model. Physical knowledge can be used to rigorously describe the system using appropriate spatial basis functions, and the spatiality of time can be decoupled. The system model can be obtained by solving 1-order ordinary differential equations (ODEs), however, these techniques require known PDEs and produce extremely high-order ODEs, which are difficult to apply in actual production.

[0005] For unknown DPS, it can be divided into two types, one is that the model structure is known but the parameters are unknown; the other is that the model structure is also unknown. When the model structure is known, the least squares method or other traditional optimization algorithms can usually be used to estimate the parameters. This actually turns into a traditional parameter estimation problem. When the model structure is unknown, the model structure and parameters need to be estimated. There is no standard solution in this case. The data-driven modeling method based on Karhunen-Loève (KL) decomposition has become one of the most commonly used methods because of its simplicity and convenience. This method first establishes a finite space basis function based on experimental data to generate a low-order ODE model; then the neural network modeling is implemented. The KL method is a linear modeling method that ignores the influence of nonlinear spatial dynamics. Therefore, the KL method can only be used to model linear or weakly nonlinear systems, and has poor effect on the modeling of strongly nonlinear DPS. Except for the recently developed KL-fuzzy method, KL-Gaussian method, spatiotemporal least squares support vector machine, and spatiotemporal recurrent neural network model, almost all data-driven modeling methods are not very effective for modeling complex nonlinear DPS. Therefore, there is currently a lack of effective modeling methods for nonlinear and complex DPS.

[0006] Due to its powerful feature representation ability, deep learning methods have attracted more and more attention from scholars in various data-driven modeling, and have shown great modeling potential for complex nonlinear DPS modeling. For example, deep learning methods represented by long short-term memory (LSTM) have good feature extraction capabilities for temporal features, and deep learning methods represented by convolutional long short-term memory (ConvLSTM) have good feature extraction capabilities for spatial-temporal features. For DPS, especially large-scale industrial processes, in order to monitor the entire system, not only a large number of sensors with spatiotemporal characteristics need to be installed on large devices, but also a large number of sensors are required to monitor various operating variables in the process, which results in DPS having multi-source spatiotemporal characteristics. However, existing methods often ignore the multi-source characteristics of process data, which invisibly hinders the improvement of model accuracy. Summary of the invention

[0007] In order to solve the above problems, the present invention provides a distributed parameter system state prediction method based on multi-source spatiotemporal information. The method comprises the following steps:

[0008] (1) Data conversion

[0009] Assuming that the measurement points of large equipment are all on the same plane, the measurement information of the equipment is viewed in the form of an image; since the reaction tubes in each device have measurement points, the reaction tubes are arranged from left to right to form a two-dimensional feature map; for each moment, a two-dimensional view of the equipment measurement points is obtained, and as time goes by, a three-dimensional spatiotemporal view of the equipment measurement points is obtained, expressed as X = (x1, x2, ..., x t) indicates that for sequence prediction, the data X of length k is sliced, which can be restated as:

[0010]

[0011] In the case of no spatial relationship but with temporal relationship, such process variables are collected to enrich the information of the model, which is expressed as U = (u1,u2,...,u t ), for sequence prediction, slicing is also required, expressed as

[0012]

[0013] (2) Temporal feature extraction

[0014] For the time series variable U s The feature extraction includes two steps: first, standardization, and second, using a single-layer LSTM network to extract time series features;

[0015] (3) Spatiotemporal feature extraction

[0016] For the time series variable X s The feature extraction mainly includes two steps. The first is standardization, which is expressed as:

[0017]

[0018] Where: xs i,std represents the i-th spatiotemporal data after standardization; xs i represents the i-th spatiotemporal data; m x represents the mean of spatiotemporal data; s x Represents the standard deviation of spatiotemporal data;

[0019] Secondly, the residual convolutional LSTM network is used to extract spatiotemporal features;

[0020] (4) The temporal features extracted in step (2) are fused with the spatiotemporal features extracted in step (3), which can be expressed as:

[0021]

[0022] in Represents the complex spatiotemporal features extracted, yes The vector representation of ;

[0023] (5) Construct a multi-source spatio-temporal network (MS-STN) model.

[0024] The fused feature data is compressed through another layer of LSTM network. Compress to the same dimension as x; finally, use the residual convolution ConvLSTM network for state prediction;

[0025] The above five steps constitute a multi-source spatiotemporal network model;

[0026] (6) The original time series data and spatiotemporal data obtained online in real time are transformed in step (1) and input into the established multi-source spatiotemporal network model to predict the process status in real time.

[0027] Preferably, in step (1), t is divisible by k. When t is not divisible by k, t / k takes the maximum number divisible by k.

[0028] Preferably, the standardization process in step (2) is expressed as:

[0029] Among them: us i,std represents the i-th time series data after standardization; us i represents the i-th time series data; m u Represents the mean of time series data; s u Represents the standard deviation of the time series data.

[0030] Preferably, the network structure used for spatiotemporal feature extraction in step (3) has more than 30 layers.

[0031] Preferably, the residual convolutional LSTM network used in the step (3) is used for modeling by a residual block composed of a ConvLSTM and a standardization combination in order to overcome the gradient vanishing / gradient exploding problem. The residual block consists of a residual part and a direct mapping part. The residual part consists of two layers of ConvLSTM and standardization; the direct mapping part consists of one ConvLSTM and standardization.

[0032] Preferably, in step (3), the residual network does not have an activation function, and a residual convolutional network is constructed by more than 10 residual blocks.

[0033] Preferably, binary cross entropy is used as the loss function in step (5).

[0034] Compared with the prior art, the present invention has the following advantages:

[0035] In order to solve the modeling problem of a distributed parameter system with multi-source spatiotemporal characteristics, the present invention constructs a multi-source spatiotemporal network model, which extracts process variable information with time series characteristics based on a residual convolutional LSTM network and extracts spatiotemporal variable information with spatiotemporal characteristics based on a ConvLSTM network. Comprehensively utilizing various process information is conducive to improving the fitting accuracy of the model and can also improve the generalization ability of the model. In order to prevent the gradient from disappearing or exploding due to the increase in the number of network layers when extracting complex spatiotemporal data features, a residual network model structure based on ConvLSTM is proposed during model training. Since the residual network can avoid model degradation, the established model can fully extract the spatiotemporal characteristics of the process. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] The present invention will be further described below in conjunction with the accompanying drawings.

[0037] Figure 1 It is a flow chart of DPS modeling based on multi-source spatiotemporal information fusion of the present invention;

[0038] Figure 2 It is a schematic diagram of the Res-ConvLSTM block of the present invention. DETAILED DESCRIPTION

[0039] The technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Therefore, the following detailed description of the embodiments of the present invention provided in the drawings is not intended to limit the scope of the claimed invention, but merely represents selected embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without making creative work belong to the scope of protection of the present invention.

[0040] Implementation Case 1

[0041] The method mainly includes six steps:

[0042] (1) Use data transformation to preprocess the collected historical data, that is, data conversion.

[0043] To simplify the problem, it is assumed that the measurement points of large equipment are all on the same plane, so that the measurement information of the equipment can be viewed in the form of an image. Specifically, since each reaction tube has a measurement point, the reaction tubes are arranged from left to right to form a two-dimensional feature map. For each moment, a two-dimensional view of the equipment measurement point can be obtained. As time goes by, a three-dimensional spatiotemporal view of the equipment measurement point can be obtained, which can be expressed as X = (x1, x2, ..., x t). For sequence prediction, it is necessary to slice the data X of length k, which can be restated as:

[0044]

[0045] It is assumed here that t is divisible by k. If this is not the case, t / k is the largest number that is divisible by k.

[0046] In addition, in the absence of spatial relationships, process variables related to the main monitoring indicators of the equipment are considered. These variables have no spatiotemporal relationship, but have a temporal relationship. These variables are collected to enrich the information of the model, expressed as U = (u1, u2, ..., u t ). For sequence prediction, slicing is also required, expressed as:

[0047]

[0048] (2) Construct a LSTM-based time series feature network to extract time series features.

[0049] For the time series variable U s The feature extraction mainly includes two steps. The first is standardization, which is expressed as:

[0050]

[0051] Among them: us i,std represents the i-th time series data after standardization; us i represents the i-th time series data; m u Represents the mean of time series data; s u Represents the standard deviation of the time series data.

[0052] Secondly, a single-layer LSTM network is used to extract time series features.

[0053] (3) Construct a spatiotemporal feature extraction network based on residual ConvLSTM to extract spatiotemporal features.

[0054] For the time series variable X s The feature extraction mainly includes two steps. The first is standardization, which is expressed as:

[0055]

[0056] Where: xs i,std The i-th spatiotemporal data after standardization; xs i represents the i-th spatiotemporal data; m x represents the mean of spatiotemporal data; s x Represents the standard deviation of spatiotemporal data.

[0057] Secondly, the residual convolutional LSTM network is used to extract spatiotemporal features.

[0058] The residual convolutional LSTM network is a residual network composed of a series of residual blocks (Res-ConvLSTM block), each of which consists of two subcomponents, including ConvLSTM and normalization. It should be noted that due to the complexity of spatiotemporal data, it is recommended to use a deeper network structure (more than 30 layers) to extract spatiotemporal features. Represents the extracted complex spatiotemporal features.

[0059] To overcome the gradient vanishing / exploding, a residual block consisting of a combination of ConvLSTM and normalization is used for modeling, such as Figure 2 As shown, where F and G are the residual extraction function and the direct mapping extraction function, respectively. is the input spatiotemporal data, is the data output after being processed by the residual block. The residual block consists of the residual part ( Figure 2 The left part) consists of two layers of ConvLSTM and normalization; the direct mapping part ( Figure 2 The right part) consists of a ConvLSTM and normalization. Thus, a residual convolutional LSTM network with shortcut connections can be constructed, so that the input signal can be directly transmitted from any low layer to the high layer. Since the output of ConvLSTM has been activated by the tanh function, the residual network here does not have an activation function. Figure 2 The residual blocks can build a residual convolutional LSTM network. More than 10 residual blocks are used here.

[0060] (4) The extracted temporal features are integrated with the spatiotemporal features, which can be expressed as

[0061]

[0062] in yes The vector representation of .

[0063] (5) Constructing a multi-source spatiotemporal network model (MS-STN)

[0064] The fused features are passed through another LSTM network for feature compression. Compress to the same dimension as x. Finally, the residual convolution ConvLSTM network is used for state prediction to establish a multi-source spatiotemporal network model. Note that binary cross entropy is used as the loss function here.

[0065] (6) The original time series data and spatiotemporal data obtained online in real time are transformed in step (1) and input into the established multi-source spatiotemporal network model (MS-STN) to predict the process state in real time.

[0066] Implementation Case 2

[0067] In order to verify the effectiveness of the present invention, data of industrial ethylene oxychlorination reactor were collected for verification. There are many factors that affect the ethylene oxychlorination reaction, including reaction temperature, reaction pressure, raw material ratio, purity of raw gas, residence time, etc. For the reaction temperature, since the process is a strong exothermic reaction, increasing the temperature is beneficial to the reaction, but if the temperature is too high, the amount of trichloroethane produced as a byproduct increases and the selectivity of the reaction decreases. Therefore, the reaction temperature is generally controlled at about 180-320°C; for the reaction pressure, since the oxychlorination method uses air, the reaction gas contains a large amount of inert gas, and in order to keep the reaction gas at a considerable partial pressure, pressurization operation is often used. For the raw material ratio, a slightly excessive amount of C2H4 and about 50% excess O2 are generally used to completely convert HCI; for the purity of the raw gas, the content of acetylene, propylene and C4 olefins should be strictly controlled; for the residence time, it needs to be controlled within a reasonable range so that the conversion rate of HCI is close to complete conversion. Among the many process variables, the reactor temperature is a particularly important variable. Due to the large scale of the reactor equipment, many temperature measurement points need to be installed at different locations in the equipment to provide an accurate view of the reactor temperature status.

[0068] The ethylene oxychlorination reactor in this example is equipped with five reaction tubes equipped with temperature measurement sensors. Each reaction tube has five fixed temperature measurement points for measuring the temperature of the reaction. In addition, for the reaction process, there are 29 process variables related to the reaction process, which is a typical multi-source spatiotemporal system. In this experiment, a total of 8,000 samples were collected, of which the first 5,000 were used for training and the last 3,000 were used for testing. It should be noted that in order to train the model, the data needs to be transformed, and the time step is set to 10, so the training set has 500 samples and the test set has 300 samples.

[0069] In order to verify the effectiveness of the method, all methods use a 40-layer network structure, using binary cross entropy as the loss function, Adadelta as the optimization function, the learning rate is 0.1, the batch size is 40, the validation set split rate is 0.2, and the early stopping technique is used to prevent overfitting. In res-ConvLSTM, the filter is set to 10, the kernel size is set to (3, 3), and the step size is set to (1, 1). As a comparison, all data are ignored as spatial relationships and regarded as spatiotemporal data, and the LSTM method is used for simulation modeling, wherein the number of network layers is 40, the number of hidden layer latent variables is 10, and the number of other parameters is consistent with the method of the present invention. The ConvLSTM method ignores the process variables of the time characteristics and only uses the temperature data of the spatiotemporal characteristics for modeling. Its network layer number is 40, the filter is set to 10, the kernel size is set to (3, 3), the step size is set to (1, 1), and the number of other parameters is consistent with the method of the present invention.

[0070] As can be seen from Table 1, the fitting and prediction results of the method of the present invention are the best (small errors in the training set and the test set), which further illustrates the superiority of the method of the present invention. It is worth mentioning that from the perspective of the root mean square error index, the average prediction error of the method proposed by the present invention is less than 1°C, indicating that the method of the present invention can effectively predict temperature changes and provide a strong guarantee for monitoring the temperature state of the reactor.

[0071] Table 1 Comparison of the prediction performance of different methods by binary cross entropy and root mean square error indicators

[0072]

[0073] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A distributed parameter system state prediction method based on multi-source spatiotemporal information, Features: The following steps are involved: (1) Data conversion Assuming that the measurement points of large equipment are all on the same plane, the measurement information of the equipment is viewed in the form of an image; since each reaction tube has a measurement point, the reaction tubes are arranged from left to right to form a two-dimensional feature map; for each moment, a two-dimensional view of the equipment measurement point is obtained, and as time goes by, a three-dimensional spatiotemporal view of the equipment measurement point is obtained, and X=(x 1 , x 2 , ..., x t ) indicates that for sequence prediction, the data X of length k is sliced, which can be restated as: In the case of no spatial relationship but with temporal relationship, such process variables are collected to enrich the information of the model, which is expressed as U = (u 1 ,u 2 , ..., u t ), for sequence prediction, slicing processing is also required, expressed as: (2) Temporal feature extraction For the time series variable U s The feature extraction includes two steps: first, standardization, and second, using a single-layer LSTM network to extract time series features; (3) Establish a spatiotemporal feature extraction network based on residual convLSTM For the time series variable X s The feature extraction mainly includes two steps. The first is standardization, which is expressed as Where: xs i,std represents the i-th spatiotemporal data after standardization; xs i represents the i-th spatiotemporal data; m x represents the mean of spatiotemporal data; s x Represents the standard deviation of spatiotemporal data; Secondly, the residual convolutional LSTM network is used to extract spatiotemporal features; The residual convolutional LSTM network used, in order to overcome the gradient vanishing / exploding, a residual block composed of ConvLSTM and normalization is used for modeling. The residual block consists of a residual part and a direct mapping part. The residual part consists of two layers of ConvLSTM and normalization; the direct mapping part consists of one ConvLSTM and normalization. (4) Fusion of extracted temporal features and spatiotemporal features The temporal features extracted in step (2) are fused with the spatiotemporal features extracted in step (3), expressed as: in Represents the complex spatiotemporal features extracted, yes The vector representation of is the time series feature; (5) Constructing a multi-source spatiotemporal model MS-STN The fused feature data is compressed through a layer of LSTM network. L s Compress to the same dimension as x; finally, use the residual convolution ConvLSTM network to predict the spatiotemporal state of the future process to establish a multi-source spatiotemporal network model; (6) The original time series data and spatiotemporal data obtained online in real time are transformed in step (1) and input into the established multi-source spatiotemporal network model MS-STN to predict the process state in real time.

2. According to claim 1, a distributed parameter system state prediction method based on multi-source spatiotemporal information, Features: In the step (1), t is divisible by k. When t is not divisible by k, t / k is the maximum number divisible by k.

3. According to claim 1, a distributed parameter system state prediction method based on multi-source spatiotemporal information, Features: The standardization process in step (2) is expressed as: Among them: us i,std represents the i-th time series data after standardization; us i represents the i-th time series data; m u Represents the mean of time series data; s u Represents the standard deviation of the time series data.

4. According to claim 1, a distributed parameter system state prediction method based on multi-source spatiotemporal information, Features: The network structure used for spatiotemporal feature extraction in step (3) has more than 30 layers.

5. According to claim 1, a distributed parameter system state prediction method based on multi-source spatiotemporal information, Features: In step (3), the residual network does not add an activation function, and the residual convolutional network is built by more than 10 residual blocks.

6. A distributed parameter system state prediction method based on multi-source spatiotemporal information according to claim 1, Features: In the step (5), binary cross entropy is used as the loss function.