A multivariate asynchronous time series data classification method and device

Abstract

The present invention discloses a method for classifying multivariate asynchronous time series data and proposes a novel time-aware dual-attention and memory-enhanced network. By combining attention and memory, it effectively simulates the complex interactions in irregular, asynchronous, and sparsely sampled multivariate asynchronous time series data, designs a randomly generated interpolation module, and constructs a dual-task learning system to achieve the coordinated unification of missing value interpolation and classification networks. In the process of interpolating and classifying missing values ​​in asynchronous time series data, the present invention can effectively mine the implicit correlations in the data, mine meaningful latent semantics, and make full use of the auxiliary information in the sequence data to discover meaningful patterns from unlabeled data. At the same time, considering the interaction between the two tasks of interpolating and classifying missing values ​​in asynchronous time series data helps to alleviate the error propagation problem, capture mutual influences, and improve the accuracy of the model.

Description

Technical Field

[0001] The present invention relates to the field of time series data anomalies, and in particular to a multivariate asynchronous time series data classification method and device. Background Art

[0002] Irregular, asynchronous, sparsely sampled multivariate asynchronous time series data (IASS-MTS) can be generated in various fields in the real world, such as climate science, ecology, astronomy, finance, and medicine. The data characteristics are as follows: Figure 2 Specifically, the figure shows (a) irregularity: different observation intervals within the same time series, for example, the intervals between observation times t1 and t2 and between observation times t2 and t3 are different; (b) asynchronousity: different sampling rates for different time series, for example, the sampling rate of time series x1(t) is greater than that of x1(t); and (c) sparseness: time series with large sampling intervals. This type of time series data is often generated in applications where the observation process is somewhat restricted or prohibits regular observation / sampling. For example, in air pollution forecasting, the collected multivariate asynchronous time series data is often: 1) incomplete due to sensor damage, data transmission failures, or storage corruption; and 2) asynchronous when collected from channels with different time scales. These characteristics of IASS-MTS data pose significant challenges to classic machine learning models and algorithms, which require data to be defined in a coherent fixed-dimensional feature space with a constant interval between consecutive time steps.

[0003] In recent years, domestic and foreign literature has made some progress in the study of IASS-MTS. Neil et al. proposed Phased-LSTM, which adds a new time gate k tTo extend LSTM units to handle irregularly sampled time series, improving the performance of existing RNN models. Che et al. proposed the GRU-D model based on the Gated Recurrent Unit (GRU). This model, combined with irregular time intervals, addresses the asynchronous sampling characteristics from the perspective of missing data. The GRU-D model augments the input with a binary response indicator and simulates the attenuation mechanism of irregularly sampled data. Shukla and Marlin proposed the Interpolated Prediction Network (IPNet) framework, which applies several semi-parametric interpolation schemes to obtain regularly sampled time series representations using multiple interpolation functions. Horn et al. employed a set function-based approach to classify irregularly sampled and misaligned time series observations. Other work has also addressed the issue of handling irregularities in IASS-MTS data. While these works have addressed the issues with IASS-MTS data to some extent, several challenges remain: 1) Data correlations measured by IASS-MTS often exist both within and between time series. Existing methods mostly focus on intra-series correlations, while inter-sequence interactions are poorly learned. It is worth noting that different observation sequences are often intertwined, and these inter-sequence relationships are often valuable information for downstream tasks. For example, a patient's blood pressure at a specific time is not only correlated with blood pressure at other times, but also has a strong relationship with heart rate at that time or other times. 2) Most methods propose other requirements that may not apply to real IASS-MTS data. For example, many methods process data with low missing rates, which makes them unable to perform well when the missing rate is large or continuous missing values ​​appear, that is, when the data is highly sparse. 3) Existing research on classifying irregularly sampled time series mostly does not consider auxiliary information. When the amount of existing training data is large, using only supervised data is not enough to learn reasonable parameters. Summary of the Invention

[0004] In order to solve the above technical problems existing in the prior art, the present invention proposes a multivariate asynchronous time series data classification method and device, the specific technical solution of which is as follows:

[0005] A multivariate asynchronous time series data classification method includes the following steps:

[0006] Step 1: Use formulas to define the format of multivariate asynchronous time series data in actual application scenarios;

[0007] Step 2: In the dimension-dependent sequence attention embedding module, the learned time embedding is used to provide an interpolated set, learn a fixed-length latent representation for the multivariate asynchronous time series, and obtain a set of interpolated univariate time series;

[0008] Step 3: In the modality-indicator-enhanced inter-sequence attention embedding module, all interpolated univariate time series are taken as input. The attention mechanism is used to learn the relationship between different univariate time series at different time steps. The outputs of the attention module are stacked as the final learned representation.

[0009] Step 4: In the external dynamic memory module, learn a parameterized memory matrix to cache global time information and obtain the read vector at each time point;

[0010] Step 5: In the interpolation network module, a set of latent random variables is introduced, and the Bayesian method is used in the variational framework to reconstruct the original multivariate asynchronous time series information and interpolate the missing values ​​of the unobserved time of the original multivariate asynchronous time series;

[0011] Step 6: In the dual-task learning module, the classification model is trained by optimizing the objective to obtain reliable classification results and accurately reconstruct the time series.

[0012] Furthermore, the step 1 is specifically as follows: formulating and defining the representation form of the multivariate asynchronous time series data, and setting the multivariate asynchronous time series data to Represents N data instances, where each data instance contains a D-dimensional irregular, asynchronous, and sparsely sampled multivariate asynchronous time series s n ={s n,d |d=1,...,D}, whose label is y n , s n Each dimension in is a univariate time series s n,d , d represents the dimension, each univariate time series s n,d For a list of observation tuples:

[0013]

[0014] Where T n,d yes The number of observations of the d-th variable in the n-th data instance, is a multivariate asynchronous time series The d-th univariate in the n-th instance of n,d The observation value of , the corresponding observation time is The timestamp information is listed in Indicates that the observation value of the d-th single variable in the n-th data instance is listed express.

[0015] Furthermore, the step 2 specifically includes the following sub-steps:

[0016] Step 2.1, pre-select L relevant time points as interpolation time, and encode the interpolation time and observation time; specifically, in the dimension-related sequence, the attention embedding module pre-selects L relevant time points τ = [τ1, τ2, ..., τ L ] as the interpolation time, and the time point τ l ∈[τ1,τ2,...,τ L ], observation time value set t n,d and the observed value x n,d,j As input, the code is obtained at time point τ l The output vector The specific formula is as follows:

[0017]

[0018] Among them, γ(τ l ,t n,d,j ) is the interpolation weight;

[0019] In step 2.2, the interpolation weights of the multivariate asynchronous time series are calculated using the attention mechanism to learn a fixed-length latent representation for the multivariate asynchronous time series and obtain a set of interpolated univariate time series. Specifically, the expression of the interpolation weights is:

[0020]

[0021] Interpolation weight γ(τ l ,t n,d,j ) by the query matrix and the key value matrix After softmax normalization, is the normalization factor, ψ(τ) is the time encoding vector of time τ, and the time encoding method is as follows:

[0022]

[0023] ψ(τ)[i] is the i-th dimension of ψ(τ), are learnable parameters. This encoding process uses a sine function to capture periodic patterns in the data and a linear term to represent time and capture non-periodic behavior in the time series.

[0024] therefore, The interpolated time series of the dth variable in the nth data instance is obtained, x n =[x n,1 ,....,x n,D ] is the set of interpolated univariate time series of the nth data instance.

[0025] Furthermore, the step three specifically includes the following sub-steps:

[0026] Step 3.1: add variable identifiers to each variable and learn their embedding vectors to distinguish multiple univariate time series. Use local position embedding technology to learn and obtain the order information of interpolated univariate time series. Specifically, in the inter-sequence attention embedding module, first add and learn an embedding vector ε of the variable identifier d. d , used to distinguish multiple univariate time series, and use local position embedding technology to learn the order information of interpolated univariate time series, the lth position embedding vector The calculation formula is as follows:

[0027]

[0028] where l∈{1,...,L} is the local position,

[0029] Step 3.2: Integrate the interpolated univariate sequence, variable identifier embedding vector, and local position embedding vector, input the attention mechanism, learn the relationship between the sequences, and obtain the final embedding expression; Specifically, sum the interpolated univariate sequence, variable identifier embedding vector, and local position embedding vector to obtain the final input z of the attention mechanism. n :

[0030]

[0031] The inter-sequence attention function IAF is as follows:

[0032]

[0033] Use multi-head attention, where W Q ,W K ,W V is a trainable parameter, and then passes through a fully connected feedforward network and a residual network complete attention module based on a conventional attention mechanism, and splices and stacks multiple results to obtain the final output vector, which is the multidimensional time series u n :

[0034]

[0035] in

[0036] Furthermore, the step 4 specifically includes the following sub-steps:

[0037] Step 4.1, the vector of the multidimensional time series at the current time point is spliced ​​with the memory read vector set of the previous time point and input into the neural network controller to obtain the interface vector; specifically, in the external dynamic memory module, the multidimensional time series u n The vector value at time point l and the memory read vector set {r n,1 ,...,r n,l-1} are spliced ​​and input into a neural network controller (NN controller) to obtain the interface vector ξ n,l ;

[0038] Step 4.2, use the interface vector to calculate and update the memory matrix, and obtain the reading vector at each time point; specifically, the interface vector ξ n,l Divide into interface parameter calculation to generate write vector v n,l , write vector weights Read vector weights and the erasure vector e n,l , used to calculate and update the memory matrix The expression is as follows:

[0039]

[0040] By calculation Returns the lth read vector r n,l , and finally concatenate L read vectors to obtain the vector representation of all time steps

[0041] Furthermore, the step five is specifically as follows: in the interpolation network module, a set of latent random variables are introduced, and the Bayesian method is used to reconstruct the multivariate asynchronous time series information, and the cyclic self-encoder-decoder and the attention module within the codec sequence are combined in the variational framework to interpolate the unobserved time missing values ​​of the original multivariate asynchronous time series.

[0042] Furthermore, the interpolation network module includes a recognition network and a generation network;

[0043] In the recognition network, the original multivariate asynchronous time series is interpolated using the intra-coding sequence attention module to obtain a latent representation of the time series with a fixed length, which is then input into the recurrent autoencoder to obtain the recognition model.

[0044] In the generative network, the latent random variables are sampled and input into the recurrent self-decoder together with the latent representation of the fixed length of the original time series. Then, after the attention module in the decoding sequence, the latent representation of the fixed length of the original time series is mapped back to the original observation space to obtain the generation conditional probability of the original space data.

[0045] Furthermore, the step six specifically includes the following sub-steps:

[0046] Step 6.1: Concatenate the outputs of the dimension-dependent intra-sequence attention embedding module, the inter-sequence attention embedding module, and the external dynamic memory module as the final potential representation of the time series and input it into the classifier to obtain the classification result;

[0047] Step 6.2, obtain the lower bound of the maximum likelihood estimate of the original multivariate asynchronous time series at the variational lower bound value level from the output of the interpolation network module;

[0048] In step 6.3, the classification cross entropy loss function and the lower bound of the maximum likelihood estimate of the original multivariate asynchronous time series at the variational lower bound value level are used as the loss function of the classification model, and the classification model is optimized using the stochastic gradient descent algorithm.

[0049] A multi-element asynchronous time series data classification device is characterized by comprising one or more processors for implementing the multi-element asynchronous time series data classification method.

[0050] A computer-readable storage medium is characterized in that a program is stored thereon, and when the program is executed by a processor, it implements the multivariate asynchronous time series data classification method.

[0051] Beneficial effects:

[0052] In the process of interpolating and classifying missing values ​​of asynchronous time series data, the present invention can effectively mine the implicit correlations in the data, explore meaningful potential semantics, and make full use of the auxiliary information in the sequence data to discover meaningful patterns from unlabeled data. At the same time, considering the interaction between the two tasks of interpolating and classifying missing values ​​of asynchronous time series data, it helps to alleviate the error propagation problem, capture mutual influence, and improve the accuracy of the model. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] Figure 1 This is a flow chart of a multivariate asynchronous time series data classification method of the present invention;

[0054] Figure 2 It is a waveform diagram showing the specific characteristics of the multivariate asynchronous time series data IASS-MTS;

[0055] Figure 3 This is the overall architecture diagram of the memory-enhanced network model on which the method of the present invention is based;

[0056] Figure 4a This is a comparison chart of the experimental results of the method of the present invention and the existing methods on the PhysioNet dataset;

[0057] Figure 4b This is a comparison chart of the experimental results of the method of the present invention and the existing methods on the Human Activity dataset;

[0058] Figure 5a This is a comparison chart of the experimental results of the method of the present invention and the existing method after interpolation data completion on the Toy dataset;

[0059] Figure 5b This is a comparison chart of the experimental results of the proposed method and the existing method on the MuJoco Physics dataset after interpolation data completion;

[0060] Figure 6 Schematic diagram of ablation experiment results of various mechanism modules adopted by the method of the present invention;

[0061] Figure 7 It is a structural schematic diagram of a multivariate asynchronous time series data classification device of the present invention. DETAILED DESCRIPTION

[0062] In order to make the purpose, technical solution and technical effect of the present invention more clear, the present invention is further described in detail below with reference to the accompanying drawings and embodiments.

[0063] The present invention proposes a multivariate asynchronous time series data classification method, which adopts a new dual temporal attention mechanism and memory enhancement network model (DAMA-Net). By combining attention and memory, it effectively simulates the complex interactions in irregular, asynchronous, and sparsely sampled multivariate asynchronous time series data (IASS-MTS), captures the temporal irregularity, multiple sampling rates, and global temporal pattern information in the IASS-MTS multivariate asynchronous time series, and designs a random generation interpolation module. By introducing the random generation interpolation network, the memory enhancement network model is further extended, which is called the DAMA-NetSGI model. The overall framework of the model is as follows: Figure 3 As shown in the figure, unlike the memory-augmented network model, the DAMA-NetSGI model adopts dual-task learning to build a dual-task learning system, unifying the interpolation network and the classification network to achieve the coordinated unification of missing value interpolation and classification networks, and solve the asynchronous interaction and irregular and sparse sampling problems of IASS-MTS multivariate asynchronous time series data.

[0064] The DAMA-NetSGI model has the following three major improvements:

[0065] First, by combining random generative interpolation network learning to properly interpolate input data, the effectiveness of time series classification is improved;

[0066] Second, considering the limited amount and availability of training data, an unsupervised learning component is added to the DAMA-Net model to learn reasonable parameters of the randomly generated interpolation network, while also improving the representation ability of the attention module;

[0067] Third, the DAMA-NetSGI model combines advances in deep generative models with stochastic interpolation networks for recurrent time series models. This model can capture the variability of observed time series data and can be used to reconstruct incomplete original input sequences or estimate missing patterns in time series for regression tasks. Based on these three key improvements, extensive experimental results and analysis are presented to demonstrate the effectiveness of the proposed model.

[0068] Specifically, such as Figure 1 As shown, the method of the present invention comprises the following steps:

[0069] Step 1: Define the multivariate asynchronous time series data format for actual application scenarios.

[0070] Specifically, we define the representation of multivariate asynchronous time series data by formulating it into a machine-readable data representation. Represents N data instances, where each data instance contains a D-dimensional irregular, asynchronous, and sparsely sampled multivariate asynchronous time series s n ={s n,d |d=1,...,D}, whose label is y n , s n Each dimension in is a univariate time series s n,d , each univariate time series s n,d For a list of observation tuples: Where T n,d yes The number of observations of the d-th variable in the n-th data instance, is a multivariate asynchronous time series The d-th univariate in the n-th instance of n,d The observation value of , the corresponding observation time is Here, timestamp information is listed in Indicates that the observation value of the d-th single variable in the n-th data instance is listed For multivariate asynchronous time series data, different single-dimensional vectors of multivariate asynchronous time series have different observation time numbers, that is, in general, for any two different dimensions d and d', there is t n,d ≠t n,d′ , T n,d ≠T n,d′ For a given irregularly sampled multivariate time series The problem is defined as: 1) learning a mapping model F:S→Y by estimating the function The probability distribution of the sequence is used to predict the category y∈{0,1} C, where C is the number of categories; 2) Reconstruct the original sequence p(x|s) from the missing time series.

[0071] Step 2: In the dimension-dependent sequence attention embedding module, the learned time embedding is used to provide an interpolation set, learn a fixed-length latent representation for irregularly and asynchronously sampled multivariate asynchronous time series, and obtain an interpolated univariate time series set. The specific steps include the following:

[0072] Step 2.1: preselect L relevant time points as interpolation time, and encode the interpolation time and observation time;

[0073] Specifically, in the intra-series attention embedding module, L relevant time points τ = [τ1, τ2, ..., τ L ] as the interpolation time, and the time point τ l ∈[τ1,τ2,...,τ L ], observation time value set t n,d and the observed value x n,d,j As input, the code is obtained at time point τ l The output vector The specific formula is as follows:

[0074]

[0075] Among them, γ(τ l ,t n,d,j ) is the interpolation weight;

[0076] Step 2.2: Use the attention mechanism to calculate the interpolation weights of the multivariate asynchronous time series, learn a fixed-length latent representation for the multivariate asynchronous time series, and obtain a set of interpolated univariate time series.

[0077] Specifically, the expression of the interpolation weight is:

[0078] Interpolation weight γ(τ l ,t n,d,j ) by the query matrix and the key value matrix After softmax normalization, is the normalization factor, ψ(τ) is the time encoding vector of time τ, and the time encoding method is as follows:

[0079]

[0080] ψ(τ)[i] is the i-th dimension of ψ(τ), are learnable parameters. This encoding process uses a sine function to capture periodic patterns in the data and a linear term to represent time and capture non-periodic behavior in the time series.

[0081] therefore, The interpolated time series of the dth variable in the nth data instance is obtained, x n =[x n,1 ,....,x n,D ] is the set of interpolated univariate time series of the nth data instance.

[0082] Step 3: In the modality-indicator-enhanced inter-sequence attention embedding module, all interpolated univariate time series are taken as input. The attention mechanism is used to learn the relationship between different univariate time series at different time steps. The output of the attention module is stacked as the final learned representation. The specific steps include the following:

[0083] Step 3.1, Modality IND: Add variable identifiers to each variable and learn their embedding vectors to distinguish multiple univariate time series. Use local position embedding technology to learn and obtain the order information of interpolated univariate time series.

[0084] Specifically, in the inter-series attention embedding module, a variable identifier d is first added, and the embedding vector ε of the variable identifier d is learned. d , used to distinguish multiple univariate time series, and use the local position embedding (LOCAL PE) technology to learn the order information of the interpolated univariate time series. The lth position embedding vector The calculation formula is as follows:

[0085]

[0086] where l∈{1,...,L} is the local position,

[0087] Step 3.2: Integrate the interpolated univariate sequence, variable identifier embedding vector, and local position embedding vector, input the attention mechanism, learn the relationship between sequences, and obtain the final embedding expression;

[0088] Specifically, the interpolated univariate sequence, the variable identity embedding vector and the local position embedding vector are summed to obtain the final input z of the attention mechanism n :

[0089]

[0090] The inter-sequence attention function (IAF) is as follows:

[0091]

[0092] Multi-head attention can be used here, where W Q ,W K ,W V is a trainable parameter, and then passes through a fully connected feedforward network and a residual network complete attention module based on a conventional attention mechanism, and splices and stacks multiple results to obtain the final output vector, which is the multidimensional time series u n :

[0093]

[0094] in

[0095] Step 4: In the external dynamic memory module, learn a parameterized memory matrix to cache global time information and obtain the read vector at each time point. This includes the following sub-steps:

[0096] Step 4.1: concatenate the vector at the current time point of the multidimensional time series with the memory read vector set at the previous time point and input them into the neural network controller to obtain the interface vector;

[0097] Specifically, in the external dynamic memory module, the multidimensional time series u n The vector value at time point l and the memory read vector set {r n,1 ,...,r n,l-1} are spliced ​​and input into a neural network controller (NN controller) to obtain the interface vector ξ n,l ;

[0098] Step 4.2: Use the interface vector to calculate and update the memory matrix, and obtain the reading vector at each time point;

[0099] Specifically, the interface vector ξ n,l Divide into interface parameter calculation to generate write vector v n,l , write vector weights Read vector weights and the erasure vector e n,l , used to calculate and update the memory matrix, namely the memory matrix (memory matrix), the expression is as follows:

[0100]

[0101] By calculation Returns the lth read vector r n,l , and finally concatenate L read vectors to obtain the vector representation of all time steps

[0102] Step 5: In the interpolation network module, a set of latent random variables is introduced, and the Bayesian method is used in the variational framework to reconstruct the original multivariate asynchronous time series information and interpolate the missing values ​​of the unobserved time of the original multivariate asynchronous time series.

[0103] Specifically, the interpolation network module introduces a set of hidden random variables, and uses the Bayesian method to reconstruct multivariate asynchronous time series information. It combines the recurrent self-encoder-decoder (RNN) and the attention module within the encoder-decoder sequence in a variational framework, mainly including two key networks: the recognition network and the generation network.

[0104] In the recognition network, which is equivalent to the encoder, the attention module in the coding sequence is used to interpolate the original multivariate asynchronous time series, obtain the potential representation of the fixed length of the time series, and then input it into the RNN encoder to obtain the recognition model;

[0105] Specifically, in the recognition network, for each n-th data instance, that is, the original multivariate asynchronous time series s n ,in

[0106]

[0107] First, it is input into the intra-series attention embedding module (the same as the intra-series attention embedding module mentioned above) to obtain the n-th data instance interpolation univariate time series set x n =[x n,1 ,....,x n,D ],in Interpolate the time series for the dth variable in the obtained nth data instance; then use the RNN encoder to map the interpolated variable to the latent space vector z n , the specific mapping process is as follows:

[0108]

[0109] in Represents the nth data instance at time point τ l The hidden variables, Initialized to a vector of all 0s, are parameters in the neural network, ζ and κ are parameters with neural network, Indicates that the mean is μ n,z The covariance is a diagonal matrix Multivariate Gaussian distribution, identification model is the posterior probability Variational approximation of .

[0110] In the generator network, which is equivalent to the decoder, the latent random variables are sampled and input into the RNN decoder together with the latent representation of the original time series of fixed length. After the attention module in the decoded sequence, the latent representation of the original time series of fixed length is mapped back to the original observation space to obtain the generation conditional probability of the original space data.

[0111] Specifically, in the generative network, the process is opposite to that of the recognition network. First, the sampled latent vector z n,r , here, in order to make the entire model differentiable, the latent vector z n The sampling process is redefined as:

[0112]

[0113] in To assist Gaussian noise, ⊙ represents point multiplication; the latent vector is obtained through a linear mapping to obtain the initial hidden layer of the RNN decoder The RNN decoder then updates the hidden layer state through the following process:

[0114]

[0115] in are all parameters in RNN, so as to learn the hidden layer sequence The hidden layer is input into the decoding sequence intra-series attention embedding module (intra-seriesattention), which has the opposite function to the encoding sequence intra-series attention embedding module. Mapped to the set of all observation time points of the nth data instance sample Get the interpolated time series of the nth data instance sample Assuming that the generated conditional probability follows a diagonal covariance moment Gaussian distribution, the generated conditional probability distribution is parameterized by the following process:

[0116]

[0117] in are the parameters in the neural network.

[0118] Step 6: In the dual-task learning module, the classification model is trained by optimizing the objectives (accurate interpolation and correct classification) to obtain reliable classification results and accurately reconstruct the time series. This includes the following sub-steps:

[0119] Step 6.1: Concatenate the outputs of the dimension-dependent intra-sequence attention embedding module, the inter-sequence attention embedding module, and the external dynamic memory module as the final potential representation of the time series and input it into the classifier to obtain the classification result;

[0120] Specifically, the dimension-related intra-sequence attention embedding module, the modality index-enhanced inter-sequence attention embedding module, and the output x of the external dynamic memory module are embedded in the n 、u n and r n Perform concatenation to obtain the final potential representation of the time series: And put it into the classifier g θ (·) Get the classification The loss function is obtained as:

[0121]

[0122] Where C is the number of categories, is the probability that the data may belong to c, α is the weight adjustment parameter, β is the focus parameter, {θ * All learnable parameters of the model.

[0123] Step 6.2, obtain the lower bound of the maximum likelihood estimate of the original multivariate asynchronous time series at the variational lower bound value level from the output of the interpolation network module;

[0124] Specifically, by maximizing the variational lower bound ELBO:

[0125]

[0126] To maximize the maximum likelihood estimate Substituting the output of the interpolation network module into the formula, we can obtain:

[0127]

[0128] In step 6.3, the categorical cross entropy loss function and the variational lower bound are used as the loss function of the model, and the stochastic gradient descent algorithm (SGD) is used to optimize the model.

[0129] Specifically, SDG is used to optimize the model by optimizing the following composite objective function:

[0130]

[0131] Where λ=0.1 is a predefined parameter.

[0132] This paper conducts experiments on synthetic and real datasets to qualitatively and quantitatively verify the effectiveness of the proposed DAMA-NetSGI model in irregularly sampled time series classification and interpolation data completion tasks. This paper uses the Toy, PhysioNet, Human Activity and MuJoco Physics datasets for experiments. On the PhysioNet and HumanActivity datasets, experiments on classification and interpolation problems are conducted. By comparing with 11 existing methods, it is verified that this method works best when processing IASS-MTS data. The experimental results are shown in Figure 2. Figure 4a and 4b As shown; on the Toy and MuJoco Physics datasets, the invention has performed interpolation data completion tasks, and the experimental results are as follows Figure 5a and 5b As shown in the figure, the experiment shows the effectiveness of the method proposed by the invention. In addition, the present invention compares the contribution of each module through ablation experiments. The experimental results are as follows: Figure 6 As shown, the results show that intra-sequence attention mechanism, inter-sequence attention mechanism and dynamic memory are necessary, and other modules also promote data classification.

[0133] In summary, the main contributions of the present invention are summarized as follows:

[0134] (1) We introduce a sequence-dependent intra-sequence attention embedding module associated with learned temporal encodings, which takes multivariate asynchronous time series data as input and produces a fixed-length latent representation in a set of interpolations that encapsulate intra-sequence interactions and circumvent asynchronous sampling features.

[0135] (2) Based on the intra-sequence attention embedding network, the present invention constructs an inter-sequence attention module, adds a morphology (sequence) indicator and a position embedder to the sequence, and adopts an external memory module as DAMA-Net to capture the global temporal dynamics, thereby alleviating the inherent sparsity of real multivariate asynchronous time series data and improving the prediction performance;

[0136] (3) This paper designs a random interpolation network based on DAMA-Net to capture the variability observed in time series data, further learns high-performance parameters to enhance the representation ability of the attention module, and reconstructs the input sequence to estimate the missing pattern of the time series, thereby improving the effect of time series classification;

[0137] (4) This paper proposes a dual-task learning mechanism to collaboratively optimize multiple objectives, using multi-level sequential information and temporal information to jointly train downstream task modules with complementary losses, thereby producing reliable classification and interpolation results;

[0138] (5) Detailed comparative tests and ablation experiments were conducted on the DAMA-NetSGI model to verify the effectiveness of the model.

[0139] Corresponding to the aforementioned embodiment of a multi-element asynchronous time series data classification method, the present invention also provides an embodiment of a multi-element asynchronous time series data classification device.

[0140] See also Figure 7 An embodiment of the present invention provides a multivariate asynchronous time series data classification device, including one or more processors, for implementing the multivariate asynchronous time series data classification method in the above embodiment.

[0141] An embodiment of a multi-element asynchronous time series data classification device of the present invention can be applied to any device with data processing capabilities, and the device with data processing capabilities can be a device or apparatus such as a computer. The device embodiment can be implemented through software, or through hardware or a combination of software and hardware. Taking software implementation as an example, as a device in a logical sense, it is formed by the processor of any device with data processing capabilities in which it is located reading the corresponding computer program instructions in the non-volatile memory into the memory for execution. From the hardware level, if Figure 7 As shown in the figure, it is a hardware structure diagram of any device with data processing capability where a multi-element asynchronous time series data classification device of the present invention is located. Figure 7 In addition to the processor, memory, network interface, and non-volatile memory shown, any device with data processing capabilities in which the apparatus in the embodiment is located may also include other hardware, generally based on the actual functions of the device with data processing capabilities, which will not be described in detail.

[0142] The implementation process of the functions and effects of each unit in the above-mentioned device is specifically described in the implementation process of the corresponding steps in the above-mentioned method, and will not be repeated here.

[0143] For the device embodiments, since they basically correspond to the method embodiments, the relevant parts can be referred to the partial description of the method embodiments. The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the present invention. A person of ordinary skill in the art can understand and implement the present invention without inventive work.

[0144] An embodiment of the present invention further provides a computer-readable storage medium having a program stored thereon. When the program is executed by a processor, a multivariate asynchronous time series data classification method in the above embodiment is implemented.

[0145] The computer-readable storage medium may be an internal storage unit of any device with data processing capabilities described in any of the aforementioned embodiments, such as a hard disk or memory. The computer-readable storage medium may also be an external storage device, such as a plug-in hard disk, a smart media card (SMC), an SD card, a flash card, etc. equipped on the device. Furthermore, the computer-readable storage medium may also include both an internal storage unit and an external storage device of any device with data processing capabilities. The computer-readable storage medium is used to store the computer program and other programs and data required by any device with data processing capabilities, and may also be used to temporarily store data that has been output or is to be output.

[0146] The above description is only a preferred embodiment of the present invention and does not constitute any form of limitation to the present invention. Although the implementation process of the present invention is described in detail above, it is still possible for those familiar with the art to modify the technical solutions described in the above examples or to replace some of the technical features therein with equivalents. Any modifications, equivalent replacements, etc. made within the spirit and principles of the present invention shall be included in the scope of protection of the present invention.

Claims

1. A multivariate asynchronous time series data classification method, characterized in that: The following steps are involved: Step 1: Use formulas to define the format of multivariate asynchronous time series data in actual application scenarios; Step 2: In the dimension-dependent sequence attention embedding module, the learned time embedding is used to provide an interpolated set, learn a fixed-length latent representation for the multivariate asynchronous time series, and obtain a set of interpolated univariate time series; Step 3: In the modality-indicator-enhanced inter-sequence attention embedding module, all interpolated univariate time series are taken as input. The attention mechanism is used to learn the relationship between different univariate time series at different time steps. The outputs of the attention module are stacked as the final learned representation. Step 4: In the external dynamic memory module, learn a parameterized memory matrix to cache global time information and obtain the read vector at each time point; Step 5: In the interpolation network module, a set of latent random variables is introduced. The original multivariate asynchronous time series information is reconstructed using the Bayesian method in the variational framework. The recurrent self-encoder-decoder and the intra-encoder-decoder sequence attention module are combined in the variational framework to interpolate the missing values ​​of the unobserved time of the original multivariate asynchronous time series. Step 6: In the dual-task learning module, the classification model is trained by optimizing the objective to obtain reliable classification results and accurately reconstruct the time series; The interpolation network module includes a recognition network and a generation network; In the recognition network, the original multivariate asynchronous time series is interpolated using the intra-coding sequence attention module to obtain a latent representation of the time series with a fixed length, which is then input into the recurrent autoencoder to obtain the recognition model. In the generative network, the latent random variables are sampled and input into the recurrent self-decoder together with the latent representation of the fixed length of the original time series. Then, after the attention module in the decoding sequence, the latent representation of the fixed length of the original time series is mapped back to the original observation space to obtain the generation conditional probability of the original space data.

2. The multivariate asynchronous time series data classification method according to claim 1, characterized in that: The step 1 is specifically as follows: formulating and defining the representation form of the multivariate asynchronous time series data, Represents N data instances, where each data instance contains a D-dimensional irregular, asynchronous, and sparsely sampled multivariate asynchronous time series s n ={s n,d |d=1,...,D}, whose label is y n , s n Each dimension in is a univariate time series s n,d , d represents the dimension, each univariate time series s n,d For a list of observation tuples: Where T n,d yes The number of observations of the d-th variable in the n-th data instance, is a multivariate asynchronous time series The d-th univariate in the n-th instance of n,d The observation value of , the corresponding observation time is The timestamp information is listed as follows: Indicates that the observation value of the d-th single variable in the n-th data instance is listed express.

3. The multivariate asynchronous time series data classification method according to claim 2, characterized in that: The second step specifically includes the following sub-steps: Step 2.1, pre-select L relevant time points as interpolation time, and encode the interpolation time and observation time; specifically, in the dimension-related sequence, the attention embedding module pre-selects L relevant time points τ = [τ1, τ2, ..., τ L ] as the interpolation time, and the time point τ l ∈[τ1,τ2,...,τ L ], observation time value set t n,d and the observed value x n,d,j As input, the code is obtained at time point τ l The output vector The specific formula is as follows: Among them, γ(τ l ,t n,d,j ) is the interpolation weight; In step 2.2, the interpolation weights of the multivariate asynchronous time series are calculated using the attention mechanism to learn a fixed-length latent representation for the multivariate asynchronous time series and obtain a set of interpolated univariate time series. Specifically, the expression of the interpolation weights is: Interpolation weight γ(τ l ,t n,d,j ) by the query matrix and the key value matrix After softmax normalization, is the normalization factor, ψ(τ) is the time encoding vector of time τ, and the time encoding method is as follows: ψ(τ)[i] is the i-th dimension of ψ(τ), are learnable parameters. This encoding process uses a sine function to capture periodic patterns in the data and a linear term to represent time and capture non-periodic behavior in the time series. therefore, The interpolated time series of the dth variable in the nth data instance is obtained, x n =[x n,1 ,....,x n,D ] is the set of interpolated univariate time series of the nth data instance.

4. The multivariate asynchronous time series data classification method according to claim 3, characterized in that: The step three specifically includes the following sub-steps: Step 3.1: add variable identifiers to each variable and learn their embedding vectors to distinguish multiple univariate time series. Use local position embedding technology to learn and obtain the order information of interpolated univariate time series. Specifically, in the inter-sequence attention embedding module, first add and learn an embedding vector ε of the variable identifier d. d , used to distinguish multiple univariate time series, and use local position embedding technology to learn the order information of interpolated univariate time series, the lth position embedding vector The calculation formula is as follows: where l∈{1,...,L} is the local position, Step 3.2: Integrate the interpolated univariate sequence, variable identifier embedding vector, and local position embedding vector, input the attention mechanism, learn the relationship between the sequences, and obtain the final embedding expression; Specifically, sum the interpolated univariate sequence, variable identifier embedding vector, and local position embedding vector to obtain the final input z of the attention mechanism. n : With n =[z n,1 ,...,With n,j ,...,With n,d ], The inter-sequence attention function IAF is as follows: Use multi-head attention, where W Q ,W K ,W V is a trainable parameter, and then passes through a fully connected feedforward network and a residual network complete attention module based on a conventional attention mechanism, and splices and stacks multiple results to obtain the final output vector, which is the multidimensional time series u n : in 5. The multivariate asynchronous time series data classification method according to claim 4, characterized in that: The fourth step specifically includes the following sub-steps: Step 4.1, the vector of the multidimensional time series at the current time point is spliced ​​with the memory read vector set of the previous time point and input into the neural network controller to obtain the interface vector; specifically, in the external dynamic memory module, the multidimensional time series u n The vector value at time point l and the memory read vector set {r n,1 ,...,r n,l-1 } are spliced ​​and input into a neural network controller to obtain the interface vector ξ n,l ; Step 4.2, use the interface vector to calculate and update the memory matrix, and obtain the reading vector at each time point; specifically, the interface vector ξ n,l Divide into interface parameter calculation to generate write vector v n,l , write vector weights Read vector weights and the erasure vector e n,l , used to calculate and update the memory matrix The expression is as follows: By calculation Returns the lth read vector r n,l , and finally concatenate L read vectors to obtain the vector representation of all time steps 6. The multivariate asynchronous time series data classification method according to claim 1, characterized in that: The step six specifically includes the following sub-steps: Step 6.1: Concatenate the outputs of the dimension-dependent intra-sequence attention embedding module, the inter-sequence attention embedding module, and the external dynamic memory module as the final potential representation of the time series and input it into the classifier to obtain the classification result; Step 6.2, obtain the lower bound of the maximum likelihood estimate of the original multivariate asynchronous time series at the variational lower bound value level from the output of the interpolation network module; In step 6.3, the classification cross entropy loss function and the lower bound of the maximum likelihood estimate of the original multivariate asynchronous time series at the variational lower bound value level are used as the loss function of the classification model, and the classification model is optimized using the stochastic gradient descent algorithm.

7. A multivariate asynchronous time series data classification device, characterized in that: The method comprises one or more processors for implementing a multivariate asynchronous time series data classification method according to any one of claims 1 to 6.

8. A computer-readable storage medium, characterized in that A program is stored thereon, and when the program is executed by a processor, a multivariate asynchronous time series data classification method according to any one of claims 1 to 6 is implemented.

Images