Global-to-local robustness multivariable time sequence representation learning method

Through the global to local robust multivariate time series list learning method, the denoising autoencoder and mask learning mechanism of GRU to GRU solves the shortcomings of existing models in capturing global correlation and noise robustness. , realizes efficient and accurate feature extraction.

CN118940816APending Publication Date: 2024-11-12HUNAN UNIV OF SCI & TECH
View PDF 0 Cites 2 Cited by

Patent Information

Application Number
CN202410994167.8
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-24
Publication Date
2024-11-12

AI Technical Summary

Technical Problem

The characterization learning model of existing multivariate time series data is difficult to capture global temporal correlation and is not robust enough to noise, resulting in low feature extraction efficiency and poor accuracy.

Method used

Using a global to local robust multivariate time series list learning method, through the denoising autoencoder and mask learning mechanism of GRU to GRU, variable independent global feature learning and variable correlation local feature learning are carried out, combined with denoising Learn and mask learning mechanisms to improve the robustness of the model and noise tolerance.

Benefits of technology

Faster feature extraction speed and higher accuracy are achieved, the stability and noise tolerance of the model are improved, and more accurate and robust multivariate time series features are generated.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118940816A_ABST
    Figure CN118940816A_ABST
Patent Text Reader

Abstract

The invention discloses a global-to-local robustness multivariable time sequence representation learning method, which comprises the following steps of: regarding global feature learning and local feature learning as two continuous learning stages, and fusing a denoising learning mechanism and a mask learning mechanism to improve the tolerance of a model to noise; specifically, in the first learning stage, variable-independent global feature learning is executed, global features are learned by constructing a self-encoding reconstruction network based on GRU to GRU and designing a global multi-scale noise sampling strategy, and overall features of a multivariable time sequence are preliminarily obtained; in the second learning stage, variable-associated local feature learning is executed, global features learned in the first stage are fused by designing a multivariable global feature aggregation strategy, and on this basis, a GRU-based bidirectional prediction network and a mask learning mechanism are constructed to generate smooth and robust local features; details are supplemented to the overall features of the multivariable time series, and the final comprehensive, accurate and robust overall features are obtained.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of multivariate time series data, and in particular to a robust multivariate time series representation learning method from global to local. Background Art

[0002] Multivariate time series (MTS) collected by heterogeneous sensors at specific time intervals present the operating status of complex production systems in industry, commerce, and other fields. However, due to the complexity of multivariate time series (MTS), each variable may have a different periodic variation pattern, and there is coupling between variables, which makes feature extraction difficult. In addition, with the development of big data and Industry 4.0, MTS contains more valuable information, more complex variable features, stronger coupling between variables, and more noise interference. Therefore, accurately extracting features from noisy MTS has become an urgent task and challenge.

[0003] The goal of representation learning is to learn accurate features of unlabeled data without a specific task orientation. Although representation learning has made significant progress in the fields of complex networks and machine vision, its application in the field of time series analysis is still in its infancy. For MTS analysis tasks, representation learning has two potential advantages. First, the learned representation vectors can replace the original MTS data as the input of downstream analysis tasks, thereby improving the performance of analysis tasks. Second, since the learned representation vectors are relatively simple and easy to identify, they can simplify the complexity of downstream task models, thereby improving the timeliness and accuracy of the models. However, MTS usually has complex and changeable characteristics, such as global and local dynamics and coupling between multiple variables, which makes MTS representation learning more challenging. Therefore, deep neural networks such as recurrent neural networks (RNNs), convolutional neural networks (CNNs), and graph attention networks (GATs) have inevitably become key technical tools for MTS representation learning.

[0004] From the perspective of feature extractors, existing MTS representation models can be simply divided into three categories.

[0005] RNN-based models. Recurrent neural network (RNN) is the first model proposed to extract temporal dependencies of time series and has been widely used. RNN usually uses recursive memory units to capture the local dependencies between each timestamp and its neighboring timestamps. For example, Rashmi combined RNN with attention mechanism to accurately capture the periodic changes of each timestamp and its neighboring timestamps. The Bi-DVRNN model attempts to combine VAE with RNN to accurately extract local features from two directions to achieve accurate anomaly detection. However, these models have difficulty in capturing the long-term dependencies of time series. Therefore, some excellent variants have been proposed to better capture the long-term dependencies of MTS, such as long short-term memory network (LSTM) and gated recurrent unit network (GRU). For example, the Hybrid ES-LSTM model takes each timestamp and its neighboring timestamps in a fixed-length lookback window as samples and learns the local features of each timestamp. When the features of each timestamp are accurately learned, the overall features of MTS can be generated. The SGRU model uses GRU combined with selective state updates and an adaptive hybrid gradient strategy to better capture the long-term dependency features of MTS. However, none of these models take into account the feature coupling of multiple variables. The METRO model treats MTS as a series of temporal dynamic graphs while using GRU to model dynamic and cross-scale variable correlations. However, for high-dimensional and complex MTS data, these RNN-based models may take a long time. In addition, there may be outliers and noise in the MTS data, which may interfere with the learning and prediction performance of RNN.

[0006] CNN-based models. This type of representation model can use convolution operations to capture local patterns and features in the sequence, and extract higher-level features by stacking multiple convolutional layers. For example, the triplet-loss model constructs a CNN representation network, in which multi-channel convolutions are used to learn local features, and local features are concatenated to form overall features by stacking multiple layers of convolution. However, this model only considers univariate time series and cannot be extended to multivariate time series. The TS2Vec model performs hierarchical contrastive learning on two enhanced context views to generate a robust representation vector for each timestamp. The SeqVAECNN model combines the variational autoencoder network (VAE) and CNN, and uses the Seq2Seq structure to better capture temporal and spatial correlations. However, these models focus more on the association between contexts and ignore the global association between timestamps. The TimesNet model uses two-dimensional convolution to quickly capture periodic changes and inter-periodic associations in MTS. Compared with RNN, CNN has stronger parallel computing capabilities and is not prone to gradient vanishing and gradient exploding problems. However, these models do not consider the noise in MTS. In addition, they all try to generate overall features by aggregating local features, which is insufficient in capturing global correlations.

[0007] Transformer-based models. This type of model has a strong feature learning capability and can easily capture fine-grained correlations between any two neighbors through a self-attention mechanism. For example, the Log-Trans model uses the Transformer as the encoder base and uses a pre-training scheme to effectively capture the local features of MTS. The TFAI model uses the Transformer architecture to capture the local features of MTS in both the time and frequency domains. The TSLANet model integrates CNN into the Transformer to extract the features of MTS in the frequency domain. To improve accuracy, the DSANet model first uses variable-independent local feature learning, and then uses the coupling of multiple variables to optimize the local features of each variable. To further improve comprehensiveness, the SageFormer model divides an MTS into multiple different cycles, learns the local features of each timestamp in each cycle, and then merges the local features of multiple cycles. Compared with RNN and CNN, this type of model does have higher parallelism and longer effective memory range. However, the structure of this type of model is more complex and requires more computing resources. In addition, this type of model also tends to accurately learn the local features of MTS, rarely considering global correlation, and also rarely considering the impact of noise.

[0008] In summary, the three feature extractors for processing MTS data have their own characteristics. CNN is simple and efficient, suitable for fast feature extraction, but it is difficult to capture long-term temporal dependencies. RNN, especially its variant GRU, is good at processing temporal dependencies, but may require a long training time. Transformer has stronger feature learning capabilities due to its self-attention mechanism, but its structure is complex and requires high computing resources.

[0009] In summary, there are still two challenges in the existing MTS representation learning model that need to be further explored.

[0010] The first challenge is that most existing models only learn local temporal correlations and ignore the learning of global temporal correlations. Specifically, existing models usually first learn the local dependencies between each timestamp and its neighboring timestamps, and then combine these local features to form an overall feature, such as Figure 1 (a). For example, LSTM-based models mostly focus on local features within a fixed window, while CNN-based models indirectly form overall features by stacking multiple layers of convolution. In short, these models usually need to sample each timestamp and generate its neighbor information. In this case, a large number of training samples will be generated, which greatly increases the training time of the model. In addition, if the local feature learning is inaccurate (such as affected by noise), the quality of the overall feature will also be affected. This will not only affect the model performance, but also reduce the training efficiency.

[0011] The second challenge is that existing models rarely consider the impact of noise and are not robust enough. Multivariate time series (MTS) data acquired by sensors are often mixed with noise, which mainly comes from sensor failures and external interference (such as vibration or collision). These noises not only affect the observation results of a specific timestamp, but may also have an adverse effect on the local features of its adjacent timestamps, thereby causing feature drift. When local features drift, the learned MTS overall features may also be deformed or become inaccurate. At present, most existing models are not compatible with noise and still show low robustness when facing noisy data. The main reason is that the existence of noise is ignored during the design, resulting in large fluctuations in model performance when facing noise, and failure to provide robust and accurate representation vectors. Summary of the invention

[0012] In response to the above-mentioned technical problems to be solved, the present invention provides a robust multivariate time series representation learning method from global to local. By using a global to local learning framework and two noise tolerance mechanisms, denoising learning and mask learning, the model has good robustness and noise tolerance.

[0013] In order to solve the above technical problems, the technical solution proposed by the present invention is:

[0014] A robust multivariate time series representation learning method from global to local, based on the principle of "drawing outlines first and then filling in details" Variable feature learning is considered as two consecutive learning stages, namely global feature learning and local feature learning, and is integrated Denoising The global feature learning stage performs variable-independent feature learning by constructing a "GRU to GRU" autoencoder reconstruction network and designing a global multi-scale noise sampling strategy; the local feature learning first designs a multivariate global feature aggregation strategy to fuse the global features learned in the global feature learning stage, and then constructs a GRU-based bidirectional prediction network and a mask learning mechanism to generate local features, so as to obtain the overall characteristics of the multivariate time series.

[0015] As a further improvement of the above technical solution:

[0016] Preferably, the method comprises the following steps:

[0017] Step S1, the global multi-scale noise sampling strategy is used in conjunction with the "GRU to GRU" autoencoder to build a GRU-based denoising and reconstruction network, perform variable-independent global feature learning, and preliminarily obtain the global features of the multivariate time series;

[0018] Step S2, fuses the global features of fixed-length historical adjacent timestamps to obtain aggregated global features, performs variable-associated local feature learning, builds a GRU-based bidirectional prediction network and mask learning mechanism, learns the local features of each timestamp, and obtains the overall features of the multivariate time series.

[0019] Preferably, the global multi-scale noise sampling strategy is specifically:

[0020] S1-1-1, Multi-scale Computing

[0021] Use the Fast Fourier Transform to transform each variable of length T Convert to a frequency component with a length of T / 2; if the T value is too large, the variable X i will be divided into multiple groups; if the T value is too small, the variable is enhanced using the nearest neighbor interpolation method; then, the amplitude of the T / 2 frequency component is calculated

[0022] Calculate the average amplitude of M variables for each frequency component Sort the average amplitudes of T / 2 frequency components from large to small and divide them into K blocks;

[0023] Select the largest amplitude in each block As the representative amplitude; calculate the scale value of K representative periodic changes shared by M variables

[0024] S1-1-2, fast global sampling

[0025] Perform the same global sampling on M variables at K scales;

[0026] First, according to the length s of the kth scale k , put the variable of length T Divide into length s k Then, the data at the same position in all groups constitute a global sample;

[0027] S1-1-3, generate global noise samples

[0028] Noise not exceeding NS% is randomly added to the generated samples to form the final global noise sample; where NS represents a preset threshold.

[0029] Preferably, the variable-independent robust global feature learning is to select GRU as the backbone, construct K "GRU to GRU" denoising autoencoders, and learn the global features of each variable; the learning processes of M variables are independent of each other and share K "GRU to GRU" denoising autoencoders; the global learning processes of K scales are executed independently and in parallel; through the learning of GRU to GRU denoising autoencoders, K global feature matrices are generated for variable i For a time series of M variables We will get K×M global feature matrices

[0030] Preferably, the global feature learning process is as follows:

[0031] S1-2-1, Sampling

[0032] By using global multi-scale noise sampling, at scale k, the variable is divided into s k Global samples, each sample contains timestamp;

[0033] S1-2-2, GRU encoder

[0034] The first global sample is created by The global feature learning process in the GRU network composed of GRU units is as follows:

[0035] Initially, the first value in the sample is input to the first GRU unit to generate d-dimensional memory information As ; then, the second value Input to the second GRU unit to generate new d-dimensional memory information As Features; continue until the feature generation of the last data in the sample;

[0036] The global sample Input to the GRU encoder, the generated global feature is

[0037] S1-2-3, GRU decoder

[0038] According to the learned global features, another The GRU network with (j+1) GRU units will serve as a decoder to reconstruct the original noise-free input at scale k. The entire reconstruction process of the GRU decoder is the same as the learning process of the GRU encoder, except for the output of the j+1th GRU unit. In the GRU decoder, the output of the (j+1)th GRU unit is:

[0039]

[0040] in, is a shared parameter used to reduce dimensionality;

[0041] S1-2-4, denoising learning

[0042] Add noise to the samples, use the noisy data for encoding and decoding, and reconstruct the original noise-free data;

[0043] S1-2-5, loss function

[0044] The mean square error function is used as the reconstruction loss:

[0045]

[0046] Where M is the number of variables, K is the number of scales, and s k is the number of global samples at the kth scale, is the number of timestamps in the global sample, represents the reconstruction value, and represents the original noise-free data.

[0047] Preferably, the step S2 specifically includes the following steps:

[0048] S2-1, Multivariate Global Feature Aggregation

[0049] Aggregate K×M global feature matrices into a single feature matrix and learn the correlation between single feature matrices;

[0050] S2-2, Local feature learning based on GRU bidirectional mask prediction

[0051] A GRU-based bidirectional mask prediction network is constructed, and the local feature learning strategy is used as a shared encoder to form a novel "1+M" structure with M bidirectional predictors to learn the association between M variables.

[0052] Preferably, the global feature aggregation process is:

[0053] Combine the K×M global feature vectors of the t-th timestamp into one vector Randomly initialize a shared total weight matrix And the total weight matrix and Z t Perform Hadamard product; then, sum the generated vector row by row to generate the global aggregate feature vector of the tth timestamp Finally, using the same W g Aggregate all timestamps to get aggregated global features

[0054] Preferably, the local feature learning process based on GRU bidirectional mask prediction is:

[0055] S2-2-1, local mask sampling

[0056] First, a local sample is generated, which includes each timestamp and its previous P-1 timestamps and the next P-1 timestamps; a multivariate time series It is divided into T local samples; two adjacent local samples will have 2P-2 timestamp overlaps; then, for each local sample, a mask label is randomly generated for at most NS% of the timestamps; finally, T local samples with mask labels are obtained;

[0057] S2-2-2, local feature learning

[0058] Based on aggregated global features Learn local features for each timestamp;

[0059] S2-2-3, bidirectional prediction based on GRU

[0060] Design a "1+M" learning structure, where local feature learning is used as a shared encoder, combined with a GRU-based bidirectional predictor to perform historical and future predictions on M variables respectively;

[0061] When the tth local sample is input into the i-th bidirectional predictor, two sets of prediction values ​​are obtained. and

[0062]

[0063] S2-2-4, mask prediction

[0064] Integrate the mask learning mechanism into the GRU-based bidirectional prediction network;

[0065] S2-2-5, loss function

[0066] The MSE function continues to be used as the loss function:

[0067]

[0068] Where T is the number of local samples, M is the number of variables, and P is the prediction length of each sample in one direction. represents the historical forecast value, represents the future forecast value, Represents the original data without mask labels.

[0069] Preferably, in the local feature learning in S2-2-2, when the t-th local sample is input, C timestamps from the (t-C+1)-th timestamp to the t-th timestamp are selected as the local neighborhood of the t-th timestamp; and then from Z g Extracting features from local neighborhoods Next, randomly initialize the shared weight matrix and with Perform Hadamard product; finally, sum the obtained matrix row by row to generate the overall eigenvector of the tth timestamp

[0070] Preferably, in the bidirectional prediction based on GRU in S2-2-3, assuming that the learning process in the "1+M" structure for the t-th local sample is as follows:

[0071] Obtain the overall feature Z through local feature learning t ; Then, Z t It will be input to M GRU-based bidirectional predictors at the same time to predict the historical values ​​from the (t-P+1)th timestamp to the tth timestamp, and the future values ​​from the tth timestamp to the (t+P-1)th timestamp;

[0072] The bidirectional prediction process of the i-th variable is as follows:

[0073] First, a two-layer perception (MLP) is used as a filter to filter the overall features. Perform filtering and dimensionality reduction to obtain the features belonging to variable i

[0074] Then, Input the first GRU unit of the history predictor / future predictor; generate the memory information of the first GRU unit through the operations of the reset gate and the update gate The historical forecast / future forecast for the tth timestamp is calculated based on h1 as follows:

[0075]

[0076] in, is a shared parameter;

[0077] Next, the prediction process is repeated until the memory information h of the Pth GRU unit is generated. P ;

[0078] Finally, according to h P The historical prediction value at the (t-P+1)th timestamp / future prediction value at the (t+P-1)th timestamp is obtained as follows:

[0079]

[0080] In summary, when the tth local sample is input into the i-th bidirectional predictor, two sets of prediction values ​​are obtained: and

[0081]

[0082] The robust multivariate time series representation learning method from global to local provided by the present invention has the following advantages compared with the prior art:

[0083] (1) The robust multivariate time series representation learning method from global to local of the present invention designs a representation learning framework from "global to local" and achieves faster speed and higher accuracy by using non-overlapping sampling, variable-independent global feature learning and variable-associated local feature learning strategies.

[0084] (2) The global-to-local robust multivariate time series representation learning method of the present invention constructs a robust representation learning model. By using a global-to-local learning framework and two noise tolerance mechanisms, denoising learning and mask learning, the model has good stability and noise tolerance. BRIEF DESCRIPTION OF THE DRAWINGS

[0085] Figure 1 (a) is a schematic diagram of local learning.

[0086] Figure 1 (b) is a schematic diagram of global learning.

[0087] Figure 2 (a) is a diagram of the noise tolerance strategy in the global feature learning process of the present invention.

[0088] Figure 2 (b) is a diagram of the noise tolerance strategy in the local feature learning process of the present invention.

[0089] Figure 3(1) is the first stage global feature learning graph in the present invention.

[0090] Figure 3 (2) is the second stage local feature learning graph in the present invention.

[0091] Figure 4 It is the global multi-scale noise sampling diagram in the present invention.

[0092] Figure 5 It is a multivariate global feature aggregation graph of timestamp t in the present invention.

[0093] FIG6(a) is a diagram illustrating local mask sampling in the present invention.

[0094] FIG6( b ) is a diagram illustrating local feature learning in the present invention.

[0095] Figure 7 It is a time series prediction performance diagram in the experimental verification of the present invention. DETAILED DESCRIPTION

[0096] The specific embodiments of the present invention are described in detail below. It should be understood that the specific embodiments described herein are only used to illustrate and explain the present invention, and are not used to limit the present invention.

[0097] A multivariate time series (MTS) is generated by continuously monitoring a dynamic system using M heterogeneous sensors at fixed time intervals. After T monitoring times, an MTS is generated, which can be represented as a matrix It includes two dimensions: variables and time. In the variable dimension, X is decomposed into Each of these represents the observation value of the i-th sensor, also called variable i. In the time dimension, X is divided into in is the set of observed values ​​of all variables M at the jth time stamp. Specifically, is the observed value of the i-th variable at the j-th time stamp.

[0098] The goal of multivariate time series representation learning is to find a nonlinear function. Map a multivariate time series X from the data space to another feature space and generate a representation vector for each timestamp. Specifically, by g θ The mapping of the original data Convert to feature representation More specifically, in Z, the jth column is the observed value of X at the jth time stamp After representation learning, Z as a feature can replace X as the input of downstream analysis tasks, thereby reducing the complexity of downstream analysis tasks and improving their performance.

[0099] The representation learning method of the present invention abandons the traditional learning idea of ​​"from local to global", follows the route of "drawing the outline first and then filling in the details", regards global feature learning (outline) and local feature learning (details) as two continuous learning stages to improve model accuracy, and integrates denoising learning and mask learning mechanisms to improve the model's tolerance to noise. Specifically: the first learning stage performs variable-independent global feature learning, and learns accurate and robust global features by building a "GRU to GRU" autoencoder reconstruction network and designing a global multi-scale noise sampling strategy to draw the "outline" of the overall characteristics of the multivariate time series; the second learning stage performs variable-associated local feature learning, and integrates the "global features" learned in the first stage by designing a multivariate global feature aggregation strategy, and on this basis, builds a GRU-based bidirectional prediction network and mask learning mechanism to generate smooth and robust local features, which fill in the "details" of the overall characteristics of the multivariate time series, and finally obtains comprehensive, accurate and robust overall features.

[0100] like Figure 1 As shown in FIG6 , the global-to-local robust multivariate time series representation learning method of the present invention specifically includes the following steps:

[0101] In step S1, the global multi-scale noise sampling strategy is used in conjunction with the "GRU to GRU" autoencoder to build a GRU-based denoising and reconstruction network to perform variable-independent robust global feature learning.

[0102] like Figure 3 (1) As shown in MTS Each variable in will undergo a robust global feature learning process independently. For variable i, the process is as follows: First, a global multi-scale noise sampling strategy is applied to generate noisy global samples at K scales. Then, in order to robustly learn the global features of variable i at K scales Construct K "GRU to GRU" denoising autoencoders. After this stage, a total of K×M global feature matrices are generated, expressed as

[0103] The specific process is:

[0104] S1-1, global multi-scale noise sampling.

[0105] An MTS usually contains multiple variables, and each variable contains multiple periodic changes, such as daily changes, weekly changes, monthly changes, and annual changes. Therefore, each variable is first sampled separately to obtain global samples that can perform independent learning of variables. In addition, in order to capture representative periodic changes in MTS, K-scale global samples are generated for each variable to enhance the comprehensiveness of global feature learning.

[0106] Secondly, traditional models usually use a fixed lookback window to generate samples for each timestamp, which results in a large overlap between samples, thus prolonging the training time. If an MTS has T timestamps, T samples will be generated, as shown in Figure 6(a). Different from traditional sampling, this embodiment divides each variable in the MTS into non-overlapping groups according to a fixed period length, such as Figure 4 In each group, the data at the same position constitutes a global sample, which reduces sample overlap, sample quantity, and ultimately shortens training time.

[0107] Then, a denoising learning mechanism is added to the global feature learning process. Figure 2 As shown in (a), slight noise is introduced into the generated global sample to create a noisy global sample. This method aims to improve the noise tolerance of the model, making it more stable and reliable in practical applications.

[0108] S1-1-1, Multi-scale Computing

[0109] Use the Fast Fourier Transform (FFT) to transform each variable of length T Convert to a frequency component with a length of T / 2. If the T value is too large, the variable X i will be divided into multiple groups. If the T value is too small, the variable is enhanced using the nearest neighbor interpolation method. Then, the amplitude of the T / 2 frequency component is calculated.

[0110] Calculate the average amplitude of M variables for each frequency component Sort the average amplitudes of the T / 2 frequency components from large to small and divide them into K blocks.

[0111] Select the largest amplitude in each block As the representative amplitude; calculate the scale value of K representative periodic changes shared by M variables

[0112] S1-1-2, fast global sampling

[0113] The same global sampling is performed on M variables at K scales, which can be performed in parallel to increase the speed.

[0114] Taking the Kth scale of the i-th variable as an example, the sampling process is as follows: First, the variable with length T is Divide into length s k Then, the data at the same position in all groups constitute a global sample. For example, the first value of all groups constitutes the first global sample of variable i Finally, the variable X i Divided into k global samples instead of the traditional T samples, thus reducing the number of samples and training time.

[0115] S1-1-3, generate global noise samples

[0116] Noise not exceeding NS% is randomly added to the generated samples to form the final global noise sample; where NS represents a preset threshold.

[0117] S1-2, “GRU to GRU” denoising autoencoder

[0118] After generating global samples, variable-independent robust global feature learning is performed. Compared with CNN and Transformer, GRU, as an advanced RNN network, has the ability to capture long-distance dependencies, and its structure is simpler than LSTM. Therefore, GRU is selected as the backbone, and K "GRU to GRU" denoising autoencoders are constructed to robustly learn the global features of each variable. The learning processes of M variables are independent of each other, but they share K "GRU to GRU" denoising autoencoders. Moreover, the global feature learning processes of K scales are also independent and can be executed in parallel.

[0119] Taking the Kth scale of variable i as an example, its global feature learning process is as follows:

[0120] S1-2-1, Sampling

[0121] By using global multi-scale noise sampling, at scale k, the variable is divided into s k Global samples, each sample contains timestamp. For example, the first global sample is

[0122] S1-2-2, GRU encoder

[0123] A GRU network consists of The encoder composed of GRU units is constructed to learn the global features at scale k. Taking the first global sample as an example, the global feature learning process in the GRU network is as follows:

[0124] Initially, the first value in the sample is input to the first GRU unit to generate d-dimensional memory information As ; then, the second value Input to the second GRU unit to generate new d-dimensional memory information As This process continues until the feature of the last data in the sample is generated.

[0125] Specifically, for the (j+1)th GRU unit:

[0126] The value of the (j+1)th and the memory information from the jth GRU unit is sent to the (j+1)th GRU unit.

[0127] First, perform the update gate operation to generate the probability vector u j+1 , to determine the need from h j How much information is retained in , which is defined as:

[0128]

[0129] in, belongs to [0~1], σ() is the sigmoid function, [] is the connection function, is a shared parameter of all GRU units.

[0130] Then, the candidate memory information of the (j+1)th value Will generate:

[0131]

[0132] in, tan() is the tangent function, ⊙ is the Hadamard product function, is a shared parameter.

[0133] Next, a reset gate operation is performed to generate another probability vector r j+1 To decide what needs to be done How much information is remembered is defined as:

[0134]

[0135] in,

[0136] Finally, the memory information h of the (j+1)th GRU unit j+1 Updated to:

[0137]

[0138] and, The d-dimensional features are used as the output of the (j+1)th GRU unit:

[0139]

[0140] In short, the global sample Input to the GRU encoder, the generated features are

[0141] S1-2-3, GRU decoder

[0142] According to the learned global features, another The GRU network with j+1 GRU units will act as a decoder to reconstruct the original noise-free input at scale K. The reconstruction process of the decoder is the same as that of the encoder, except that the output of the j+1th GRU unit is different. The specific output of the j+1th GRU unit is:

[0143]

[0144] in, is a shared parameter used for dimensionality reduction.

[0145] S1-2-4, denoising learning

[0146] To improve the noise resistance, a denoising learning mechanism is integrated into the GRU to GRU autoencoder.

[0147] Specifically, noise is first added to the samples, and then the noisy data is used for encoding and decoding to reconstruct the original noise-free data.

[0148] S1-2-5, loss function

[0149] The mean square error (MSE) function is used as the reconstruction loss:

[0150]

[0151] Where M is the number of variables, K is the number of scales, and s k is the number of global samples at the kth scale, is the number of timestamps in the global sample, represents the reconstruction value, and represents the original noise-free data.

[0152] Through the learning of GRU to GRU denoising autoencoder, K global feature matrices are generated for variable i For a time series with M variables Then K×M global feature matrices will be generated

[0153] In step S2, after variable-independent global feature learning, the outline of the overall features of the multivariate time series is generated; then, robust local feature learning of variable association is further performed to improve the details of the overall features of the multivariate time series, thereby generating the final accurate and comprehensive overall features.

[0154] In a multivariate time series, local features usually exhibit two characteristics: smoothness between adjacent timestamps and consistency with the local time evolution trend of the multivariate time series. In this embodiment, the global features of fixed-length historical adjacent timestamps are first fused to generate local features for the timestamps, thereby maintaining smoothness. Next, a GRU-based bidirectional prediction network is constructed to perform bidirectional prediction learning to ensure that the learned local features are consistent with the local evolution trend of the MTS.

[0155] In a multivariable system, there is often significant coupling between variables, which manifests as similar patterns and evolution trends. This coupling helps to learn local features. In this embodiment, the global features of multiple variables are first aggregated. Then, local feature learning is performed on the aggregated global features. Finally, a GRU prediction network with a "1+M" structure is constructed to further explore the correlation between multiple variables.

[0156] In the global feature learning stage, a denoising mechanism is used to enhance the noise robustness of the model. In the local feature learning stage, local feature learning is performed on the learned global features, so the denoising mechanism is not applicable. In this embodiment, a mask learning mechanism is used instead of a denoising learning mechanism, and it is integrated into the local feature learning stage to achieve noise tolerance, such as Figure 2 (b) as shown.

[0157] In order to perform local feature learning of variable association, a GRU-based robust bidirectional prediction network is constructed in the local feature learning stage, such as Figure 3 (2) as shown.

[0158] S2-1, Multivariate Global Feature Aggregation

[0159] After the global feature learning phase, a K×M global feature matrix is ​​generated In this embodiment, the representation learning goal is to generate a feature vector for each timestamp in X. To this end, a simple and effective aggregation strategy is proposed to aggregate K×M global feature matrices into a single feature matrix and learn the correlation between them, such as Figure 5 shown.

[0160] Taking the tth timestamp as an example, the global feature aggregation process is described as follows:

[0161] Combine the K×M global feature vectors of the t-th timestamp into one vector Then, a shared total weight matrix is ​​randomly initialized and compare it with Z t Perform element-wise multiplication (Hadamard product). Next, sum the generated vectors row by row to generate the global aggregate feature vector for the tth timestamp Finally, using the same W g Aggregate all timestamps in the same way to get the aggregated global features

[0162] S2-2, GRU-based bidirectional mask prediction

[0163] On the basis of aggregating global features, local features of each timestamp are learned to improve the details of the overall features. In addition, the learned local features need to meet five key requirements to ensure the accuracy of the overall feature details. First, the local features of each timestamp must be able to reconstruct its original data; second, be able to predict future short-term data; third, be able to reconstruct short-term historical data; fourth, be able to fit the coupling correlation between multiple variables; and fifth, be noise tolerant.

[0164] A GRU-based bidirectional mask prediction network with a "1+M" structure is constructed to perform local feature learning of variable associations, as shown in Figure 6(b). The network can perform both historical and future predictions. In addition, a local feature learning strategy is proposed as a shared encoder, which together with M bidirectional predictors forms a novel "1+M" structure to learn the associations between M variables. Moreover, a mask learning mechanism is adopted to provide noise tolerance for local feature learning.

[0165] Specifically, the local feature learning process of variable association is summarized as follows:

[0166] S2-2-1, local mask sampling

[0167] As shown in Figure 6(a), the local mask sampling process is as follows: First, generate a local sample, which includes each timestamp and its previous P-1 timestamps and the following P-1 timestamps; a multivariate time series It is divided into T local samples; two adjacent local samples will have 2P-2 timestamp overlaps. Then, for each local sample, a mask label is randomly generated for at most NS% of the timestamps. Finally, T local samples with mask labels are obtained.

[0168] Specifically, in the local sample, the feature of the central timestamp is the input of the bidirectional predictor. The expected output of the historical prediction is the value of the central timestamp and its P-1 timestamps before it, while the expected output of the future prediction is the value of the central timestamp and its P-1 timestamps after it.

[0169] S2-2-2, local feature learning

[0170] Based on aggregated global features A local feature learning strategy is proposed to learn the local features of each timestamp. The implementation method is to fuse the global features of multiple adjacent timestamps to obtain the local features of the last timestamp. This learning scheme ensures the smoothness of the local features of each timestamp.

[0171] Specifically, considering the tth sample, its local feature learning process is shown in Figure 6(b)(1). When the tth local sample is input, C timestamps from the (t-C+1)th timestamp to the tth timestamp are selected as the local neighborhood of the tth timestamp. g Extracting features from local neighborhoods Next, randomly initialize the shared weight matrix and with Perform Hadamard product (element-wise multiplication). Finally, sum the obtained matrix row by row to generate the overall feature vector of the t-th timestamp

[0172] S2-2-3, bidirectional prediction based on GRU

[0173] A "1+M" learning structure is designed, in which the local feature learning strategy is used as a shared encoder, together with the GRU-based bidirectional predictor, to perform historical and future predictions on M variables, respectively, as shown in Figure 6(b). This learning structure can capture the correlation between M variables, thereby improving the overall characteristics of the multivariate time series.

[0174] Specifically, taking the tth local sample as an example, its learning process in the "1+M" structure is described as follows:

[0175] First, the overall feature Z is obtained by learning local features t . Then, Z t It will be input to M GRU-based bidirectional predictors at the same time to predict the historical values ​​from the (t-P+1)th timestamp to the tth timestamp, and the future values ​​from the tth timestamp to the (t+P-1)th timestamp.

[0176] Furthermore, the bidirectional prediction process of the i-th variable is as follows:

[0177] First, a two-layer perception (MLP) is used as a filter to filter the overall features. Perform filtering and dimensionality reduction to obtain the features belonging to variable i

[0178] Then, Input the first GRU unit of the history predictor (or future predictor). Generate the memory information of the first GRU unit through the operations of the reset gate and update gate in equations (1)-(4) The historical prediction (or future prediction) for the tth timestamp is calculated based on h1 as follows:

[0179]

[0180] in, (or ) is a shared parameter.

[0181] Next, the prediction process is repeated until the memory information h of the Pth GRU unit is generated. P .

[0182] Finally, according to h P The historical prediction value at the (t-P+1)th timestamp (the future prediction value at the (t+P-1)th timestamp) is obtained as follows:

[0183]

[0184] In summary, when the tth local sample is input into the i-th bidirectional predictor, two sets of prediction values ​​are obtained: and

[0185]

[0186] S2-2-4, mask prediction

[0187] In order to tolerate noise, the mask learning mechanism is integrated into the GRU-based bidirectional prediction network.

[0188] Specifically, a local sample randomly generates multiple noise masks and uses the GRU bidirectional prediction network to predict the value of the mask to achieve noise tolerance.

[0189] S2-2-5, loss function

[0190] The MSE function continues to be used as the loss function:

[0191]

[0192] Where T is the number of local samples, M is the number of variables, and P is the prediction length of each sample in one direction. represents the historical forecast value, represents the future forecast value, Represents the original data without mask labels.

[0193] In this embodiment, a learning strategy from "global to local" is adopted to replace the existing strategy from "local to global", which can capture the overall characteristics of MTS more quickly and accurately. This strategy decomposes the problem into two consecutive learning stages: Figure 1 (b)), and then learn local features (such as Figure 1 (as shown in (a)).

[0194] Specifically, we first build a reconstruction network based on the gated recurrent unit (GRU) to learn global features, so as to quickly outline the overall features of MTS. Subsequently, we build a prediction network based on GRU to focus on learning local features and further improve the details of the overall features of MTS. Then, under the guidance of the "divide and conquer" idea, the first stage is to learn independent features of variables, and the second stage is to learn related features of variables. This "divide first and then combine" strategy aims to simplify the feature learning process and improve the accuracy of feature representation. Finally, as Figure 2 As shown, denoising learning and mask learning mechanisms are added to the above two learning stages to enhance the robustness of the model. The present invention proposes a novel GRU-based robust multivariate time series representation learning method from a global to local perspective.

[0195] Experimental verification

[0196] 1. Experimental Setup

[0197] 1. Comparison Model

[0198] In order to verify the performance of the global-to-local robust multivariate time series representation learning method (TSG2L for short) proposed in this paper, six SOAT models are used as comparison models, as shown in Table 1. Among them, TS2Vec, CRT and Triplet-Loss are CNN-based models, TS-TCC is a Transformer-based model, and TNC and CPC are RNN-based models.

[0199] Table 1 Comparison models

[0200]

[0201] 2. Tasks, Datasets, and Evaluation Metrics

[0202] At the same time, this embodiment selects three well-known time series analysis tasks to evaluate the model.

[0203] Time series prediction is the first task, and its purpose is to use the learned representation vector to predict time series data. In order to evaluate the performance of this task, this embodiment selects the mean square error (MSE) and the mean absolute error (MAE) as indicators. For these two indicators, the smaller their values, the better the performance of the model. In order to complete the prediction task, this embodiment selects six public multivariate time series as experimental data sets, as shown in Table 2.

[0204] Time series classification is the second task, whose goal is to classify time series data based on the learned representation vector. For this task, this embodiment uses accuracy (ACC) and average precision-recall area under the curve (AUPRC) as evaluation indicators. The value range of these two indicators is [0,1], where 1 represents perfect classification performance and 0 represents complete classification failure. In order to complete the classification task, this embodiment selects five public multivariate time series as experimental data sets.

[0205] Time series anomaly detection is the third task, which aims to identify anomalies in time series data through learned representation vectors. In order to measure the performance of the model in the anomaly detection task, this embodiment selects F1 score, accuracy (ACC) and area under the curve (AUC) as evaluation indicators. The value range of these indicators is also [0,1], where 1 means that the model performs well in anomaly detection, and 0 means that the model fails to effectively identify any anomalies. In order to complete the anomaly task, this embodiment selects five public multivariate time series as experimental data sets.

[0206] Table 2 Dataset

[0207]

[0208] 2. Performance Analysis

[0209] 1. Time series prediction task

[0210] The time series prediction task was selected as the first experimental task. The experiment of this embodiment uses ridgeregression as the prediction model. This embodiment predicts each data set five times with different step lengths to observe the performance changes of the seven models in short-term and long-term predictions. Figure 7 The prediction performance (MSE and MA E) of the seven models on six real datasets is shown.

[0211] from Figure 7In this embodiment, some conclusions can be drawn: 1) Considering all models, the TSG2L model has a significant advantage. On a specific data set, the performance of the TS2vec model is close to that of the TSG2L model. As for other models, such as TS-TCC, T NC, CRT, CPC, and Triplet-loss, their accuracy is generally low under most test conditions. For example, on the ETT-m2 data set, when the prediction step is 48, the MSE of the TSG2L model is 0.1790 and the MAE is 0.2993, while the MSE of TS2vec is 0.2515 and the MAE is 0.3582. 2) Considering different data sets, the characteristics of the data set have a great influence on the model performance. For example, when the time step is 24, the MSE of the TSG2L model on the ETTm2 data set is 0.1275, and the MSE on the Airquality data set is 0.8321. 3) Considering the time step, as the time step increases, the MSE and MAE indicators of all models show an upward trend. However, when the time step increases, the performance of the TSG2L model decreases relatively slowly, which further proves the advantage of the TSG2L model in handling long time series prediction tasks. For example, on the air quality dataset, as the time step increases from 24 to 672, the MAE of the TS2vec model increases from 0.6467 to 0.8876, while the MAE of TSG2L only increases from 0.5968 to 0.7981.

[0212] 2. Time series classification task

[0213] The time series classification task was selected as the second experimental task. In this experiment, the performance differences of different models on five data sets were tested. In addition, SVM was also used as a classifier for this task. Table 3 shows the classification performance (ACC and AUPRC) of the seven models. In addition, in order to further verify the classification performance, this embodiment selected the ArrowHead data set as a case study to intuitively display the classification results of the seven models.

[0214] It is not difficult to find similarities from the table. 1) Considering all models, the TSG2L model performs best. The TNC, TS2vec, and CPC models also perform well on some datasets, but are generally not as good as TSG2L. The Triplet-loss model has the highest accuracy on the ArrowHead dataset, but performs relatively poorly on other datasets. The CRT model generally performs poorly. On the Chinatown dataset, the TSG2L model has an AUC score of 0.9991, the highest among all models, while the second-best performing TS-TCC model has an AUPRC score of 0.9986. 2) Considering different datasets, the experimental results of different datasets vary greatly. For example, the ArrowHead dataset seems to be more difficult to classify because the accuracy of all models is relatively low. The AUPRC of the SharePriceIncrease dataset varies greatly, which may mean that the number of multiple classes in this dataset is unbalanced. Specifically, on the SharePriceIncrease dataset, the AUC scores vary widely, with the lowest being 0.4283 (Triplet-loss) and the highest being 0.6527 (TSG2L).

[0215] Table 3 Time series classification performance

[0216]

[0217] 3. Time Series Anomaly Detection

[0218] The time series anomaly detection task was selected as the third experimental task. In this experiment, the L1 distance was used to calculate the anomaly score. In this experiment, five data sets were tested and the experimental results were analyzed to verify the generalization performance and stability of the model. Table 4 shows the anomaly detection performance (F1, AUC and ACC) of the TSG2L model of this embodiment and six comparison models.

[0219] Table 4. Time series anomaly detection performance

[0220]

[0221] It is not difficult to find similar observations from the table. 1) Considering all models, the TSG2L model performs well in the anomaly detection task, especially on the MSL, SMAP, and SWaT datasets, where its F1 score and AUC value are significantly ahead of other comparison models, showing a clear advantage. On the SMD and MBA datasets, TSG2L also maintains a high accuracy and AUC value, although its F1 score is not the highest. TS2vec, TS-TCC and TNC, CPC, and Triplet-loss perform well on some datasets, but their overall performance is not as stable as TSG2L. The CRT model performs poorly on all datasets. 2) Considering different datasets, the dataset has a great impact on the performance of the model. For example, on the MSL dataset, the F1 scores of the TNC model and the Triplet-loss model are 0.7855 and 0.7831, respectively, while the F1 score of the TSG2L model is 0.7895. But on the SMD dataset, their F1 scores are 0.4204 and 0.5462 respectively, while TSG2L is 0.5775.

[0222] 4. Summary

[0223] The experimental results of three tasks show that the TSG2L model can capture the characteristics of MTS data more effectively than the comparison model. At the same time, its performance is more stable when processing different time series datasets.

[0224] The above implementation cases are only preferred embodiments of the present invention and are not intended to limit the present invention in any form. Although the present invention has been disclosed as above with preferred embodiments, they are not intended to limit the present invention. Therefore, any simple modification, equivalent changes and modifications made to the above embodiments according to the technical essence of the present invention without departing from the content of the technical solution of the present invention shall fall within the scope of protection of the technical solution of the present invention.

Claims

1. A robust multivariate time series representation learning method from global to local, characterized by: According to the principle of "drawing outlines first and then filling in details", multivariate feature learning is regarded as two continuous learning stages, including global feature learning and local feature learning, and two noise-tolerant learning mechanisms, denoising learning and mask learning, are integrated. The global feature learning stage performs variable-independent feature learning by constructing an autoencoder reconstruction network based on "GRU to GRU" and designing a global multi-scale noise sampling strategy; the local feature learning first designs a multivariate global feature aggregation strategy to integrate the global features learned in the global feature learning stage, and then constructs a GRU-based bidirectional prediction network and a mask learning mechanism to generate local features, so as to obtain the overall characteristics of the multivariate time series.

2. The method for learning robust multivariate time series representation from global to local according to claim 1, characterized in that: The method comprises the following steps: Step S1, the global multi-scale noise sampling strategy is used in conjunction with the "GRU to GRU" autoencoder to build a GRU-based denoising and reconstruction network, perform variable-independent global feature learning, and preliminarily obtain the global features of the multivariate time series; Step S2, fuses the global features of fixed-length historical adjacent timestamps to obtain aggregated global features, performs variable-associated local feature learning, builds a GRU-based bidirectional prediction network and mask learning mechanism, learns the local features of each timestamp, and obtains the overall features of the multivariate time series.

3. The method for learning robust multivariate time series representation from global to local according to claim 2, characterized in that: The global multi-scale noise sampling strategy is specifically: S1-1-1, Multi-scale Computing Use the Fast Fourier Transform to transform each variable of length T Convert to a frequency component with a length of T / 2; if the T value is too large, the variable X i will be divided into multiple groups; if the T value is too small, the variable is enhanced using the nearest neighbor interpolation method; then, the amplitude of the T / 2 frequency component is calculated Calculate the average amplitude of M variables for each frequency component Sort the average amplitudes of T / 2 frequency components from large to small and divide them into K blocks; Select the largest amplitude in each block as representative amplitude; Calculate the scale values ​​of K representative periodic changes shared by M variables S1-1-2, fast global sampling Perform the same global sampling on M variables at K scales; First, according to the length s of the kth scale k , put the variable of length T Divide into length s k Then, the data at the same position in all groups constitute a global sample; S1-1-3, generate global noise samples Noise not exceeding NS% is randomly added to the generated samples to form the final global noise sample; where NS represents a preset threshold.

4. The method for learning robust multivariate time series representation from global to local according to claim 3, characterized in that: The variable-independent robust global feature learning selects GRU as the backbone and constructs K "GRU to GRU" denoising autoencoders to learn the global features of each variable; The learning process of M variables is independent of each other and shares K "GRU to GRU" denoising autoencoders; the global learning process of K scales is independent and executed in parallel; through the learning of the "GRU to GRU" denoising autoencoder, K global feature matrices are generated for variable i For a time series of M variables We will get K×M global feature matrices 5. The method for learning robust multivariate time series representation from global to local according to claim 4, characterized in that: The global feature learning process is as follows: S1-2-1, Sampling By using global multi-scale noise sampling, at scale k, the variable is divided into s k Global samples, each sample contains timestamp; S1-2-2, GRU encoder The first global sample is created by The global feature learning process in the GRU network composed of GRU units is as follows: Initially, the first value in the sample is input to the first GRU unit to generate d-dimensional memory information As ; then, the second value Input to the second GRU unit to generate new d-dimensional memory information As Features; continue until the feature generation of the last data in the sample; The global sample Input to the GRU encoder, the generated global feature is S1-2-3, GRU decoder According to the learned global features, another The GRU network of GRU units will act as a decoder to reconstruct the original noise-free input at scale k; The entire reconstruction process of the GRU decoder is the same as the learning process of the GRU encoder, except for the output of the (j+1)th GRU unit; in the GRU decoder, the output of the (j+1)th GRU unit is: in, is a shared parameter used to reduce dimensionality; S1-2-4, denoising learning Add noise to the samples, use the noisy data for encoding and decoding, and reconstruct the original noise-free data; S1-2-5, loss function The mean square error function is used as the reconstruction loss: Where M is the number of variables, K is the number of scales, and s k is the number of global samples at the kth scale, is the number of timestamps in the global sample, represents the reconstruction value, and represents the original noise-free data.

6. The method for learning robust multivariate time series representation from global to local according to claim 4, characterized in that: The step S2 specifically includes the following steps: S2-1, Multivariate Global Feature Aggregation Aggregate K×M global feature matrices into a single feature matrix and learn the correlation between single feature matrices; S2-2, Local feature learning based on GRU bidirectional mask prediction A GRU-based bidirectional mask prediction network is constructed, and the local feature learning strategy is used as a shared encoder to form a novel "1+M" structure with M bidirectional predictors to learn the association between M variables.

7. The method for learning robust multivariate time series representation from global to local according to claim 6, characterized in that: The global feature aggregation process is as follows: Combine the K×M global feature vectors of the t-th timestamp into one vector Randomly initialize a shared total weight matrix And the total weight matrix and Z t Perform Hadamard product; then, sum the generated vector row by row to generate the global aggregate feature vector of the tth timestamp Finally, using the same W g Aggregate all timestamps to get aggregated global features 8. The method for learning robust multivariate time series representation from global to local according to claim 7, characterized in that: The local feature learning process based on GRU bidirectional mask prediction is: S2-2-1, local mask sampling First, a local sample is generated, which includes each timestamp and its previous P-1 timestamps and the next P-1 timestamps; a multivariate time series It is divided into T local samples; two adjacent local samples will have 2P-2 timestamp overlaps; then, for each local sample, a mask label is randomly generated for at most NS% of the timestamps; finally, T local samples with mask labels are obtained; S2-2-2, local feature learning Based on aggregated global features Learn local features for each timestamp; S2-2-3, bidirectional prediction based on GRU Design a "1+M" learning structure, where local feature learning is used as a shared encoder, combined with a GRU-based bidirectional predictor to perform historical and future predictions on M variables respectively; When the tth local sample is input into the i-th bidirectional predictor, two sets of prediction values ​​are obtained. and S2-2-4, mask prediction Integrate the mask learning mechanism into the GRU-based bidirectional prediction network; S2-2-5, loss function The MSE function continues to be used as the loss function: Where T is the number of local samples, M is the number of variables, and P is the prediction length of each sample in one direction. represents the historical forecast value, represents the future forecast value, Represents the original data without mask labels.

9. The method for learning robust multivariate time series representation from global to local according to claim 8, characterized in that: The process of the local feature learning strategy in S2-2-2 is as follows: Assume that when the t-th local sample is input, select C timestamps from the (t-C+1)th timestamp to the t-th timestamp as the local neighborhood of the t-th timestamp; then select C timestamps from Z g Extracting features from local neighborhoods Next, randomly initialize the shared weight matrix and with Perform Hadamard product; finally, sum the obtained matrix row by row to generate the overall eigenvector of the tth timestamp 10. The method for learning robust multivariate time series representation from global to local according to claim 8, characterized in that: In the bidirectional prediction based on GRU in S2-2-3, the learning process of the t-th local sample in the "1+M" structure is assumed to be: Obtain the overall feature Z through local feature learning t ; Then, Z t It will be input to M GRU-based bidirectional predictors at the same time to predict the historical values ​​from the (t-P+1)th timestamp to the tth timestamp, and the future values ​​from the tth timestamp to the (t+P-1)th timestamp; The bidirectional prediction process of the i-th variable is as follows: First, a two-layer perception (MLP) is used as a filter to filter the overall features. Perform filtering and dimensionality reduction to obtain the features belonging to variable i Then, Input the first GRU unit of the history / future predictor; generate the memory information of the first GRU unit through the operations of the reset gate and the update gate The historical / future forecast for the t-th timestamp is calculated based on h1 as follows: in, is a shared parameter; Next, the prediction process is repeated until the memory information h of the Pth GRU unit is generated. P ; Finally, according to h P The historical prediction value at the (t-P+1)th timestamp / future prediction value at the (t+P-1)th timestamp is obtained as follows: In summary, when the tth local sample is input into the i-th bidirectional predictor, two sets of prediction values ​​are obtained: and

Citation Information

Cited By

  • Fourier transform denoising method and system based on mixed mask strategy

    CN119669648A

  • Linear vector map data zero watermark method based on deep learning

    CN122390948A