Industrial equipment state detection method based on autoregressive graph adversarial neural network

Abstract

The present invention proposes an industrial equipment status detection method based on an autoregressive graph adversarial neural network, which mainly solves the problem that the prior art graph structure representation is inaccurate. The data feature extraction is not comprehensive, and it is easily affected by noise and pseudo-anomalies and misdetected. The steps of the present invention are: constructing a graph structure learning subnetwork to correctly learn the graph structure; using a sequence feature extraction subnetwork and a latent variable feature extraction subnetwork to comprehensively extract the features of the input space and the latent space; using an autoregressive regularization subnetwork and an adversarial regularization subnetwork to suppress overfitting. The present invention reduces the network error detection rate; the operating status of industrial equipment can be detected in real time through the trained network.

Description

Technical Field

[0001] The present invention belongs to the field of physical technology, and further relates to an industrial equipment state detection method based on autoregressive graph adversarial neural network in the field of identification technology using electronic equipment. The present invention can perform real-time detection through the state of industrial equipment in operation. Background Art

[0002] In the process of real-time monitoring of the operating status of industrial equipment, a large amount of multivariate time series data used to describe the operating status of the equipment is often generated. Learning these data can help determine whether the operating status of the equipment is normal. Automated anomaly detection technology based on deep learning technology aims to determine the time when the equipment is abnormal by analyzing multivariate time series data. Deep learning technology has incomparable advantages in the field of anomaly detection: through layer-by-layer nonlinear transformation, abstract features from low to high are extracted to achieve perfect modeling of multivariate time series, so that anomalies can be well identified. However, due to the variability and complexity of multivariate time series, existing methods can neither fully utilize the changing laws of different levels of multivariate time series, nor consider the problem of decreased accuracy caused by over-learning of pseudo-anomalies by the model.

[0003] Sun Yat-sen University disclosed an anomaly detection method based on unsupervised learning in its patent application "An active anomaly detection method for multivariate time series and related devices" (patent application number: CN 202111668759.3, application publication number: CN 114298240 A). The implementation steps of this method are: first, obtain the KPI data of the cloud native system, and use the KPI data to unsupervisedly train the anomaly detection model to obtain the unsupervised anomaly detection model. Then, when new KPI data is obtained, the anomaly score of each time series sample at the corresponding time observation point is obtained through the unsupervised anomaly detection model. According to the anomaly detection score, several time series samples are selected for labeling, and then several labeled time series samples are obtained. Finally, the unsupervised anomaly detection model is trained by feedback of the label information of each time series sample to obtain the final anomaly detection model for anomaly detection. There are two shortcomings in this method: First, the method has the problem of incomplete feature modeling. It ignores the joint use of sequence features and latent variable features, which leads to the lack of representation of key information of multivariate time series by the model. Second, this method does not take into account the overfitting problem caused by pseudo-anomaly learning in the reconstruction model. The overfitting problem causes the reconstruction error of the model for abnormal data to be very close to the reconstruction error of normal data, resulting in the model being unable to distinguish normal data from abnormal data.

[0004] Hang Zhao et al. published a multivariate time-series anomaly detection method based on graph attention network in their paper "Multivariate Time-series Anomaly Detection via Graph Attention Network" (IEEE International Conference on DataMining. November 2020). The specific steps of this method are: first, the time series data is divided into several time windows, and then the graph attention network is used to simultaneously extract the features of the multivariate time series in the time domain and the graph domain. Finally, the reconstruction task and the prediction task are jointly trained. There are two shortcomings in this method: First, the method has the problem of inaccurate prior assumptions. It is a priori assumed that there is a strong correlation between different time series in the multivariate time series, which misleads the model's representation of the graph domain features. Second, this method focuses on the reconstruction of sequence features and ignores the problem of model overfitting, that is, the model shows good modeling capabilities for both abnormal data and normal data, thus deviating from the goal of the model's weak modeling capabilities for abnormal data and strong modeling capabilities for normal data. Summary of the invention

[0005] The purpose of the present invention is to address the deficiencies of the above-mentioned prior art and propose an industrial anomaly detection method based on autoregressive graph adversarial neural network, aiming to solve the problems of inaccurate and incomplete feature extraction and model overfitting in multivariate time series anomaly detection.

[0006] The technical idea for achieving the purpose of the present invention is: the present invention establishes a graph structure learning subnetwork. The graph structure learning subnetwork adopts an adaptive learning strategy to characterize the interaction between time series, because this adaptive learning strategy abandons incorrect prior assumptions and uses neural networks to learn the dependencies between time series, so as to accurately describe the graph domain features. The present invention uses a sequence feature extraction subnetwork and a latent variable feature extraction subnetwork based on a graph attention layer and a gated recurrent unit layer. The sequence feature extraction subnetwork and the latent variable feature extraction subnetwork extract the features of the input data and the features of the latent space respectively, thereby avoiding the problem of incomplete feature extraction in the prior art. The present invention uses an autoregressive regularization subnetwork and an adversarial regularization subnetwork to introduce regularization technology to suppress the problem of model overfitting. The overfitting problem often means that pseudo-anomalies misguide the model optimization process, and the regularization technology reduces the problem of incorrect detection of industrial equipment status by adding regularization terms to the loss function to standardize the optimization process of the model.

[0007] The specific steps of implementing the method of the present invention include the following:

[0008] Step 1: Build a graph structure learning sub-network:

[0009] Build a graph structure learning subnetwork consisting of two fully connected layers in parallel, where the input and output channels of the two fully connected layers are set to M, where M represents the number of types of industrial equipment indicators;

[0010] Step 2: Construct a sequence feature extraction subnetwork:

[0011] Build a sequence feature extraction subnetwork consisting of a graph attention layer, a first gated recurrent unit layer, and a second gated recurrent unit layer in series; set the input and output channels of the graph attention layer to M, the input channel of the first gated recurrent unit layer to M, and the output channel to 40; set the input channel of the second gated recurrent unit layer to 40, and the output channel to M;

[0012] Step 3: Construct a latent variable feature extraction subnetwork:

[0013] Build a latent variable feature extraction subnetwork consisting of a graph attention layer and a gated recurrent unit layer in series; set the input and output channels of the graph attention layer to M, and the input channel of the gated recurrent unit layer to M, and the output channel to 40;

[0014] Step 4: Construct an autoregressive regularized subnetwork:

[0015] Build an autoregressive regularized subnetwork consisting of a gated recurrent unit layer and a fully connected layer in series; set the input channel of the gated recurrent unit layer to 40 and the output channel to M; set the input and output channels of the fully connected layer to M;

[0016] Step 5: Construct the adversarial regularization sub-network:

[0017] Build an adversarial regularization subnetwork consisting of a gated recurrent unit layer and a fully connected layer in series; set the input channel of the gated recurrent unit layer to M, the output channel to 40, and the input channel of the fully connected layer to 40, and the output channel to 1;

[0018] Step 6: Construct an autoregressive graph adversarial neural network:

[0019] An autoregressive graph adversarial neural network is constructed, wherein a graph structure learning subnetwork, a sequence feature extraction subnetwork, and a latent space feature extraction subnetwork are sequentially connected in series, and the first and second gated recurrent units in the sequence feature extraction subnetwork are respectively connected in series with the autoregressive regularization subnetwork and the adversarial regularization subnetwork;

[0020] Step 7, generate training set:

[0021] Step 7.1, collect historical indicator data of M types of industrial equipment indicators, including water quality, water volume, and water pressure, of the outlet pipe network and distribution pipe network equipment of a water supply plant; each indicator data has a duration of 1440 minutes and a sampling interval of 1 second, and is time series data that does not contain abnormal situations;

[0022] Step 7.2, normalize each time series data respectively; divide each normalized time series data into 7200 time series blocks with a duration of 12 seconds to form a training set;

[0023] Step 8: Train the autoregressive graph adversarial neural network:

[0024] Step 8.1, initialize the network weights, and initialize all weights in the network to random values ​​that satisfy the normal distribution, where the mean of the normal distribution is 0 and the standard deviation is 0.02;

[0025] Step 8.2, input the generated time series blocks of the training set into the network in sequence, and calculate the loss value of the autoregressive graph adversarial neural network; use the gradient descent method to back-propagate the total loss value of the network's loss function, and iteratively update the network parameters until the trained autoregressive graph adversarial neural network is obtained;

[0026] Step 9: Check the device status:

[0027] Step 9.1, collect M types of indicator data of water quality, water volume, and water pressure of the outlet pipe network and distribution pipe network equipment of a water supply plant in real time, and collect each time series block with a length of 12 seconds; use the same method as step 6.2 to normalize each time series block;

[0028] Step 9.2, input the normalized time series block into the trained autoregressive graph adversarial neural network to obtain the reconstructed time series block corresponding to the time series block;

[0029] Step 9.3, calculate the anomaly score of the reconstructed time series block;

[0030] Step 10: Determine the device operating status based on the device parameter judgment result:

[0031] If the abnormal score of each reconstructed time series block is greater than 0.5, the parameter of the device corresponding to the reconstructed time series block is judged to be abnormal, otherwise, the parameter of the device is judged to be normal.

[0032] Compared with the prior art, the present invention has the following advantages:

[0033] First, the present invention adopts an adaptive graph structure learning sub-network. The graph structure learning sub-network does not use inaccurate prior assumptions but adaptively learns the graph structure according to the characteristics of the time series data generated by industrial equipment, which overcomes the defect of the prior art that the model learns the wrong graph domain features due to inaccurate prior assumptions. This enables the present invention to correctly characterize the graph domain features and improves the accuracy of the model's industrial equipment status detection.

[0034] Second, since the present invention uses sequence features and latent variable features together, by simultaneously using the sequence feature extraction subnetwork and the latent variable feature extraction subnetwork to extract the features of the time series input space and latent space, the dependency information of different dimensions of the multivariate time series is effectively utilized, overcoming the deficiency of the prior art in the lack of representation of time series features. This enables the present invention to comprehensively extract features at different levels of the time series and improve the industrial equipment status detection performance of the model.

[0035] Third, since the present invention adopts regularization technology, the adverse effects of noise and pseudo-anomalies on model optimization are suppressed by using adversarial regularization subnetworks and autoregressive regularization subnetworks respectively, so that the model is optimized in the right direction. The disadvantage of the prior art that the over-learning of noise and pseudo-anomalies causes the decrease in abnormality discrimination ability, so that the present invention avoids the problem of false detection of industrial equipment status detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1 is a flow chart of the present invention; DETAILED DESCRIPTION

[0037] The present invention will be further described below in conjunction with the accompanying drawings and embodiments.

[0038] Reference Figure 1 The implementation steps of the present invention are further described with reference to the accompanying drawings and embodiments.

[0039] Step 1: Build a graph structure learning sub-network.

[0040] Build a graph structure learning subnetwork consisting of two fully connected layers in parallel. The input and output channels of the two fully connected layers are both set to 51.

[0041] Step 2: Construct a sequence feature extraction subnetwork.

[0042] Build a sequence feature extraction subnetwork consisting of a graph attention layer, a first gated recurrent unit layer, and a second gated recurrent unit layer in series; the sequence feature extraction module takes the current time series block as input, outputs hidden variables at the first gated recurrent unit layer, and outputs the sequence block reconstructed from the current time series block at the second gated recurrent unit layer; set the input and output channels of the graph attention layer to 51. Set the input channel of the first gated recurrent unit layer to 51, and the output channel to 40. Set the input channel of the second gated recurrent unit layer to 40, and the output channel to 51.

[0043] Step 3: Construct a latent variable feature extraction subnetwork.

[0044] Build a latent variable feature extraction subnetwork consisting of a graph attention layer and a gated recurrent unit layer in series; the latent variable takes the sequence block reconstructed from the current time series block as input and outputs the reconstructed latent variable. Set the input and output channels of the graph attention layer to 51. Set the input channel of the gated recurrent unit layer to 51 and the output channel to 40.

[0045] Step 4: Construct an autoregressive regularized sub-network.

[0046] Build an autoregressive regularized subnetwork consisting of a gated recurrent unit layer and a fully connected layer in series; the autoregressive regularized subnetwork takes the latent variable output in the sequence feature space as input and outputs the predicted value of the next time series block; set the input channel of the gated recurrent unit layer to 40 and the output channel to 51. The input and output channels of the fully connected layer are both set to 51.

[0047] Step 5: Construct the adversarial regularization sub-network.

[0048] Build an adversarial regularization subnetwork consisting of a gated recurrent unit layer and a fully connected layer in series; the adversarial regularization subnetwork takes the reconstructed sequence block of the sequence feature extraction module as input and outputs the category prediction value; set the input channel of the gated recurrent unit layer to 51 and the output channel to 40. Set the input channel of the fully connected layer to 40 and the output channel to 1.

[0049] Step 6: Construct an autoregressive graph adversarial neural network.

[0050] An autoregressive graph adversarial neural network is constructed, in which the graph structure learning subnetwork, the sequence feature extraction subnetwork, and the latent space feature extraction subnetwork are connected in series in sequence, and the first and second gated recurrent units in the sequence feature extraction subnetwork are connected in series with the autoregressive regularization subnetwork and the adversarial regularization subnetwork, respectively.

[0051] Step 7: Generate a training set.

[0052] Step 7.1, in the embodiment of the present invention, 51 types of historical indicator data including water quality, water volume, and water pressure of the outlet pipe network and distribution pipe network equipment of a water supply plant are collected, and the indicator details are shown in Table 1. The duration of each indicator data is 1440 minutes and the sampling interval is 1 second, which is time series data without abnormal conditions.

[0053] Table 1. List of historical indicator data for 51 categories

[0054]

[0055]

[0056]

[0057] Step 7.2: Normalize each time series data as follows:

[0058]

[0059] in, represents the i-th time series data, represents the minimum value of the i-th time series data, Indicates the maximum value of the i-th time series data. i Represents the normalized i-th time series data.

[0060] Each normalized time series data is divided into 7200 time series blocks with a duration of 12 seconds to form a training set.

[0061] Step 8: Train the autoregressive graph adversarial neural network.

[0062] Step 8.1, initialize network weights. Initialize all weights in the network to random values ​​that satisfy the normal distribution, where the mean of the normal distribution is 0 and the standard deviation is 0.02.

[0063] Step 8.2, input the generated time series blocks of the training set into the network in sequence, and calculate the loss value of the autoregressive graph adversarial neural network. Using the gradient descent method, the loss value of the network is back-propagated, and the network parameters are iteratively updated until the loss function of the network converges, and a trained autoregressive graph adversarial neural network is obtained.

[0064] The total loss value L of the loss function of the autoregressive graph against the neural network is as follows:

[0065] L=L seq +L lat +L arg +L adv

[0066] Among them, Lseq represents the loss value of the time series block input to the network after the sequence feature extraction sub-network, L lat represents the loss value of the time series block input to the network through the latent variable feature extraction sub-network, L arg Represents the loss value of the time series block input to the network after the autoregressive regularization sub-network, L adv Represents the loss value of the adversarial regularized sub-network for the time series block input to the network.

[0067] The L seq , L lat , L arg , L adv They are obtained by the following formulas:

[0068]

[0069]

[0070]

[0071]

[0072] Where N represents the total number of time series blocks in the training set, t represents the sequence number of the time series block, and ||·|| 2 represents the 2-norm operation, ∑ represents the summation operation, log represents the logarithm operation with base 2, and x (t) represents the t-th time series block, represents the reconstructed time series block of the tth time series block output by the sequence feature extraction subnetwork, express The output of the adversarial regularized sub-network, z (t) represents the latent variable of the t-th time series block output by the sequence feature extraction subnetwork, represents the reconstructed latent variable of the t-th time series block output by the latent variable feature extraction subnetwork, Represents the predicted value of the t+1th time series block output by the autoregressive regularization subnetwork.

[0073] Step 9: Check the device status.

[0074] Step 9.1, real-time collection of 51 types of indicator data, including water quality, water volume, and water pressure, from the outlet pipe network and distribution pipe network equipment of a water supply plant, with each collection time series block lasting 12 seconds. Each time series block is normalized using the same method as step 6.2.

[0075] Step 9.2, input the normalized time series block into the trained autoregressive graph adversarial neural network to obtain the reconstructed time series block corresponding to the time series block.

[0076] Step 9.3, calculate the anomaly score of the reconstructed time series block using the following formula:

[0077]

[0078]

[0079] Where k represents the sequence number of the input time series block, x (k) represents the kth time series block of the network input, represents the reconstructed time series block of the kth time series block output by the network, represents the reconstruction error value between the kth input time series block and the reconstructed time series block, express The minimum value in express The maximum value in s (k) represents the anomaly score of the kth reconstructed time series block.

[0080] Step 10: Determine the device operating status based on the device parameter judgment result.

[0081] If the abnormal score of each reconstructed time series block is greater than 0.5, the parameter of the device corresponding to the reconstructed time series block is judged to be abnormal, otherwise, the parameter of the device is judged to be normal.

[0082] The following is a simulation experiment to further illustrate the technical effect of the present invention:

[0083] 1. Simulation conditions.

[0084] The simulation experiment of the present invention was completed on a computer with an Intel(R) Core(TM) i7-7800X CPU@3.50GHz processor and an Nvidia(R) RTX2080Ti graphics card using the deep learning framework Pytorch software.

[0085] The public standard data set SWAT was used to generate the training set and test set in the simulation experiment of the present invention. The data set includes a training set of 10080 minutes and a test set of 5760 minutes. Both the training set and the test set contain 51 industrial status indicators and the sampling interval is 1s. The training set does not contain abnormal conditions of industrial equipment, and 12% of the test set has abnormal conditions of industrial equipment, and the rest are normal.

[0086] 2. Analysis of simulation content and results.

[0087] The simulation experiment of the present invention adopts the present invention and an existing technology, uses the training set in the SWAT data set, trains the autoregressive graph adversarial neural network constructed by the present invention and the network constructed by the existing technology, and obtains the trained network. The test set in the SWAT data set is input into the trained network, and the anomaly detection score of each time series block of the test set is output.

[0088] In the simulation experiment, an existing technology used is the multivariate time series anomaly detection algorithm based on graph attention network proposed by Hang Zhao et al. in “Multivariate Time-series Anomaly Detection via Graph Attention Network” (IEEE International Conference on Data Mining. November 2020).

[0089] In order to verify the simulation effect of the present invention, the precision and recall of the anomaly detection score of each time series block are calculated using the following formula, and all the calculation results are plotted in Table 2.

[0090]

[0091]

[0092] Table 2. Comparison of precision and recall of anomaly detection scores of the present invention and existing methods

[0093]

[0094] It can be seen from Table 2 that the precision of the present invention is 0.87 and the recall is 0.96. Both indicators are higher than those of the prior art method, which proves that the present invention can achieve higher industrial equipment status detection accuracy.

[0095] The above simulation experiments show that the present invention uses the constructed graph structure learning subnetwork to effectively learn the accurate graph structure, and uses the constructed sequence feature extraction subnetwork and latent variable feature extraction subnetwork to characterize the features of data in different dimensions, effectively solving the problem of incomplete feature extraction in the prior art methods, and uses the constructed autoregressive regularization subnetwork and adversarial regularization subnetwork to effectively improve the precision of industrial operation status detection, which is a very practical industrial equipment status detection method.

Claims

1. A method for detecting the state of industrial equipment based on autoregressive graph adversarial neural network, characterized in that: Construct a graph structure learning subnetwork, jointly use the sequence feature extraction subnetwork and the latent variable feature extraction subnetwork to extract the features of the time series input space and latent space, and use the adversarial regularization subnetwork and the autoregressive regularization subnetwork to suppress overfitting; the specific steps of this method include the following: Step 1: Build a graph structure learning sub-network: Build a graph structure learning subnetwork consisting of two fully connected layers in parallel, where the input and output channels of the two fully connected layers are set to M, where M represents the number of types of industrial equipment indicators; Step 2: Construct a sequence feature extraction subnetwork: Build a sequence feature extraction subnetwork consisting of a graph attention layer, a first gated recurrent unit layer, and a second gated recurrent unit layer in series; set the input and output channels of the graph attention layer to M, the input channel of the first gated recurrent unit layer to M, and the output channel to 40; set the input channel of the second gated recurrent unit layer to 40, and the output channel to M; Step 3: Construct a latent variable feature extraction subnetwork: Build a latent variable feature extraction subnetwork consisting of a graph attention layer and a gated recurrent unit layer in series; set the input and output channels of the graph attention layer to M, and the input channel of the gated recurrent unit layer to M, and the output channel to 40; Step 4: Construct an autoregressive regularized subnetwork: Build an autoregressive regularized subnetwork consisting of a gated recurrent unit layer and a fully connected layer in series; set the input channel of the gated recurrent unit layer to 40 and the output channel to M; set the input and output channels of the fully connected layer to M; Step 5: Construct the adversarial regularization sub-network: Build an adversarial regularization subnetwork consisting of a gated recurrent unit layer and a fully connected layer in series; set the input channel of the gated recurrent unit layer to M, the output channel to 40, and the input channel of the fully connected layer to 40, and the output channel to 1; Step 6: Construct an autoregressive graph adversarial neural network: An autoregressive graph adversarial neural network is constructed, wherein a graph structure learning subnetwork, a sequence feature extraction subnetwork, and a latent space feature extraction subnetwork are sequentially connected in series, and the first and second gated recurrent units in the sequence feature extraction subnetwork are respectively connected in series with the autoregressive regularization subnetwork and the adversarial regularization subnetwork; Step 7, generate training set: Step 7.1, collect historical indicator data of M types of industrial equipment indicators, including water quality, water volume, and water pressure, of the outlet pipe network and distribution pipe network equipment of a water supply plant; each indicator data has a duration of 1440 minutes and a sampling interval of 1 second, and is time series data that does not contain abnormal situations; Step 7.2, normalize each time series data respectively; divide each normalized time series data into 7200 time series blocks with a duration of 12 seconds to form a training set; Step 8: Train the autoregressive graph adversarial neural network: Step 8.1, initialize the network weights, and initialize all weights in the network to random values ​​that satisfy the normal distribution, where the mean of the normal distribution is 0 and the standard deviation is 0.02; Step 8.2, input the generated time series blocks of the training set into the network in sequence, and calculate the loss value of the autoregressive graph adversarial neural network; use the gradient descent method to back-propagate the total loss value of the network's loss function, and iteratively update the network parameters until the trained autoregressive graph adversarial neural network is obtained; Step 9: Check the device status: Step 9.1, collect M types of indicator data of water quality, water volume, and water pressure of the outlet pipe network and distribution pipe network equipment of a water supply plant in real time, and collect each time series block with a length of 12 seconds; use the same method as step 6.2 to normalize each time series block; Step 9.2, input the normalized time series block into the trained autoregressive graph adversarial neural network to obtain the reconstructed time series block corresponding to the time series block; Step 9.3, calculate the anomaly score of the reconstructed time series block as follows: Where k represents the sequence number of the input time series block, x (k) represents the kth time series block of the network input, represents the reconstructed time series block of the kth time series block output by the network, represents the reconstruction error value between the kth input time series block and the reconstructed time series block, express The minimum value in express The maximum value in s (k) represents the anomaly score of the kth reconstructed time series block; Step 10: Determine the device operating status based on the device parameter judgment result: If the abnormal score of each reconstructed time series block is greater than 0.5, the parameter of the device corresponding to the reconstructed time series block is judged to be abnormal, otherwise, the parameter of the device is judged to be normal.

2. The industrial equipment state detection method based on autoregressive graph adversarial neural network according to claim 1 is characterized in that: The normalization of each time series data in step 7.2 is obtained by the following formula; in, represents the i-th time series data, represents the minimum value of the i-th time series data, represents the maximum value of the i-th time series data, x i Represents the normalized i-th time series data.

3. The industrial equipment state detection method based on autoregressive graph adversarial neural network according to claim 1 is characterized in that: The total loss value of the loss function of the network described in step 8.2 is as follows: L=L seq +L lat +L arg +L adv Among them, L represents the total loss value of the loss function of the autoregressive graph adversarial neural network of the time series block input to the network, L seq represents the loss value of the time series block input to the network after the sequence feature extraction sub-network, L lat represents the loss value of the time series block input to the network through the latent variable feature extraction sub-network, L arg represents the loss value of the time series block input to the network after the autoregressive regularization sub-network, L adv Represents the loss value of the time series block input to the network after the adversarial regularization sub-network; The L seq , L lat , L arg , L adv They are obtained by the following formulas: Where N is the total number of time series blocks in the training set, t is the sequence number of the time series block, ||·||2 is the 2-norm operation, ∑ is the sum operation, log is the logarithm operation with base 2, and x is (t) represents the t-th time series block, represents the reconstructed time series block of the tth time series block output by the sequence feature extraction subnetwork, express The output of the adversarial regularized sub-network, z (t) represents the latent variable of the t-th time series block output by the sequence feature extraction subnetwork, represents the reconstructed latent variable of the t-th time series block output by the latent variable feature extraction subnetwork, Represents the predicted value of the t+1th time series block output by the autoregressive regularization subnetwork.

Images