A multivariate time series signal analysis method based on multi-scale context

Abstract

The present invention provides a multivariate time series signal analysis method based on multi-scale context. First, the multivariate time series signal is subjected to time-frequency conversion and segmentation of different scales, and the segmented slice vectors are used as input vectors for network training; then, the constructed multivariate time series signal analysis network is trained, and the constructed network adopts a multi-scale context Transformer model, including a multi-level encoding module and a bidirectional feature fusion module, which can better explore the potential contextual relationship between different scales and improve the feature representation of time series signals; finally, the trained network is applied to perform time series signal analysis. The present invention uses Transformer to model the multi-scale correlation of MTS signals, which can not only maximize the ability to generate features at each single scale, but also make full use of multi-scale features to enhance the overall representation capability of the MTS analysis network.

Description

Technical Field

[0001] The present invention belongs to the technical field of time series signal analysis, and particularly relates to a multivariate time series signal analysis method based on multi-scale context. Background Art

[0002] Time series are ubiquitous in many fields, including medicine, finance, meteorology, biology, smart cities, and engineering applications. In recent years, deep learning has also achieved remarkable success in time series analysis. Most research in this field has focused on univariate time series (UTS), where each instance has a single sequence. In the real world, however, each instance typically contains multiple, inherently correlated sequences. Therefore, the analysis of multivariate time series (MTS), which have several temporally correlated consecutive observations at each timestamp, has become an important and challenging area in this field.

[0003] Transformers have demonstrated strong modeling capabilities for long-range dependencies, and as a result, have recently been introduced as advanced computational models in the field of time series analysis. Transformer networks and their variants for time series analysis typically use a self-attention mechanism to encode global features to address various challenges in time series analysis. These networks have been applied to a variety of time series tasks, such as classification, regression, forecasting, and anomaly detection. Specifically, a Transformer network typically splits the input data into fixed-size slices, then applies a linear projection to the flattened slices. To preserve the positional information of each slice, a positional encoding is added to each slice before it is fed into the encoder. The encoder consists of L standard modules, each of which is composed of layer normalization, a multi-head self-attention module, a multi-layer perceptron, and residual connections. While Transformer networks achieve significant performance gains compared to traditional networks, they are limited in processing features at a single scale for MTS analysis tasks. Typically, MTS, composed of multiple simple sequences, exhibit features at multiple temporal and spatial scales, exhibiting a hierarchical structure. Modeling using only single-scale slices fails to capture the multi-scale information in the data. To address this issue, some CNNs employ receptive fields of varying sizes and numbers to extract multi-resolution features from time series signals, providing semantic information at different levels for training. However, simply fusing information at different scales through a cascade operation ignores the underlying contextual relationships between multi-scale features. This not only limits the representational power of features at each scale but also weakens the overall representation of the input signal. The above analysis reveals that effectively leveraging multi-scale context to enhance the representation of time series signals is a key challenge in MTS analysis. Summary of the Invention

[0004] In order to overcome the shortcomings of the prior art, the present invention provides a multivariate time series signal analysis method based on multi-scale context. First, the multivariate time series signal is subjected to time-frequency conversion and segmentation of different scales, and the segmented slice vectors are used as input vectors for network training; then, the constructed multivariate time series signal analysis network is trained, and the constructed network adopts a multi-scale context Transformer model, including a multi-level encoding module and a bidirectional feature fusion module, which can better explore the potential contextual relationship between different scales and improve the feature representation of time series signals; finally, the trained network is applied to perform time series signal analysis. The present invention uses Transformer to model the multi-scale correlation of MTS signals, which can not only maximize the ability to generate features at each single scale, but also make full use of multi-scale features to enhance the overall representation capability of the MTS analysis network.

[0005] A multivariate time series signal analysis method based on multi-scale context is characterized by the following steps:

[0006] S1. Given a multivariate time series signal training set, use short-time Fourier transform to convert the time series signal into a time-frequency spectrum, and splice the time-frequency spectrum of the same instance along the Z-axis direction. The spliced ​​time-frequency spectrum is divided into three scales of 8×8, 12×12, and 16×16 to obtain several slices of three scales; then, use linear mapping to map each slice into a one-dimensional vector of length 768, and use a vector of the same length to record the position information of the slice in the original signal as its position encoding vector; all slice vectors of the same scale are spliced ​​to obtain a multidimensional vector of length 768, which is used as the training parameter vector. The position encoding vector is then superimposed on the corresponding slice vector to obtain a network training vector at the same scale;

[0007] S2. Construct a multivariate time series signal analysis network, including a multi-level encoding module and a bidirectional feature fusion module. The multi-level encoding module includes three hierarchical networks with the same structure and independent parameters. Each hierarchical network adopts the VisionTransformer architecture. The training vectors of three different scales are input into the three hierarchical networks respectively, and the feature maps at three scales are output. The bidirectional feature fusion module mainly includes a feature fusion module. The input of the feature fusion module is two different feature maps. The feature maps are first spliced. The spliced ​​feature maps pass through a convolution layer containing two convolution kernels with sizes of 1×1 and 3×3 respectively, and then pass through a convolution layer containing a convolution kernel with a size of 1×1. The fused feature maps are output by the sigmoid function. The specific processing process of the bidirectional feature fusion module is as follows: for the feature maps at three scales output by the multi-level encoding module, First, forward fusion is performed, that is, the two feature maps at the small scale and the medium scale are taken as input, and processed by the feature fusion module to obtain the fused feature map F1. Then, the feature map F1 and the feature map at the large scale output by the multi-level encoding module are taken as input, and processed by the feature fusion module to obtain the forward fused feature map F2; then, reverse fusion is performed, that is, the feature maps at the large scale and the medium scale output by the multi-level encoding module are taken as input, and processed by the feature fusion module to obtain the fused feature map F3; then, the feature map F3 and the feature map at the small scale output by the multi-level encoding module are taken as input, and processed by the feature fusion module to obtain the reverse fused feature map F4; finally, the feature map F2 obtained by forward fusion and the feature map F4 obtained by reverse fusion are taken as input, and processed by the feature fusion module to obtain the final fused feature map F5;

[0008] S3. Input the training vectors at different scales obtained in step S1 into the multivariate time series signal analysis network for network training. If the cross entropy loss is used during training, a trained classification task model is obtained; if the MSE loss is used during training, a trained regression task model is obtained.

[0009] S4. After performing time-spectrum conversion and splicing on the multivariate time series signal to be processed according to the method of step S1, the signal is input into the trained multivariate time series signal analysis network obtained in step S3. The output feature map is the final classification or regression result.

[0010] The beneficial effects of the present invention are: a simple but effective multi-scale contextual multivariate time series signal analysis method is constructed, which is dedicated to using Transformer to model the multi-scale features of MTS signals based on the data characteristics of multivariate time series signals, and explores the potential relationship between the contexts of multi-level feature maps. By transferring information between different scales, the ability of each single scale to generate features can be maximized to improve the feature expression of MTS at multiple scales. In order to aggregate multi-scale information, bidirectional learning is used to iteratively fuse adjacent scale features, in which the positive and negative directions stimulate and promote each other, and the multi-scale features can be fully utilized to enhance the output representation of the network, ultimately significantly improving the analysis effect of multivariate time series signals. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] Figure 1 This is a flow chart of a multivariate time series signal analysis method based on multi-scale context of the present invention. DETAILED DESCRIPTION

[0012] The present invention is further described below with reference to the accompanying drawings and examples. The present invention includes but is not limited to the following examples.

[0013] In order to improve the performance of the Transformer model in multivariate time series signal analysis tasks, the present invention provides a multivariate time series signal analysis method based on multi-scale context. Figure 1 As shown, the specific implementation process is as follows:

[0014] S1. Given a multivariate time series signal training set, use short-time Fourier transform to convert the time series signal into a time-frequency spectrum, and splice the time-frequency spectrum of the same instance along the Z-axis direction. The spliced ​​time-frequency spectrum is divided into three scales of 8×8, 12×12, and 16×16 to obtain several slices of three scales; then, use linear mapping to map each slice into a one-dimensional vector of length 768, and use a vector of the same length to record the position information of the slice in the original signal as its position encoding vector; all slice vectors of the same scale are spliced ​​to obtain a multidimensional vector of length 768, which is used as the training parameter vector. The position encoding vector is then superimposed on the corresponding slice vector to obtain a network training vector at the same scale.

[0015] S2. Construct a multivariate time series signal analysis network, including a multi-level encoding module and a bidirectional feature fusion module. The multi-level encoding module includes three layers of networks with the same structure and independent parameters. Each layer adopts the Vision Transformer architecture, mainly composed of layer normalization, multi-head self-attention module, multi-layer perceptron, Patch Merge and residual connection. Layer normalization is used to prevent gradient explosion and gradient vanishing; the self-attention module is used to establish long-distance spatial dependencies to solve the problem of limited receptive field of convolution kernel; the multi-layer perceptron adds a fully connected hidden layer between the output layer and the input layer, and transforms the output of the hidden layer through the activation function; Patch Merge is used for downsampling, which is used to reduce the resolution and adjust the number of channels to form a hierarchical design, while also saving a certain amount of computation; the residual connection is used to integrate intermediate features into the subsequent modules with the same resolution. The training vectors of three different scales are input into the three layers of the network respectively, and the feature maps at three scales are output.

[0016] To aggregate the multi-scale semantic information generated by the multi-level encoding modules, a bidirectional feature fusion module is used for bidirectional (forward / backward) feature fusion. This module primarily consists of two feature maps. The input to the module is two different feature maps. These feature maps are first concatenated. The concatenated feature maps are passed through a convolutional layer with two convolution kernels of 1×1 and 3×3, respectively, followed by a convolutional layer with a convolution kernel of 1×1. The fused feature maps are then output using a sigmoid function. The specific processing process of the bidirectional feature fusion module is as follows: for the feature maps at the three scales output by the multi-level coding module, first forward fusion is performed, that is, the two feature maps at the small scale and the medium scale are used as input, and are processed by the feature fusion module to obtain the fused "small medium scale" feature map F1, and then the "small medium scale" feature map F1 and the large scale feature map output by the multi-level coding module are used as input, and are processed by the feature fusion module to obtain the forward fused "small medium large scale" feature map F2; then reverse fusion is performed, that is, the large scale and medium scale feature maps output by the multi-level coding module are used as input, and are processed by the feature fusion module to obtain the fused "large medium small scale" feature map F3; then the feature map F3 and the small scale feature map output by the multi-level coding module are used as input, and are processed by the feature fusion module to obtain the reverse fused "large medium small scale" feature map F4; finally, the "small medium large scale" feature map F2 obtained by forward fusion and the "large medium small scale" feature map F4 obtained by reverse fusion are used as input, and are processed by the feature fusion module to obtain the final fused feature map F5. The bidirectional feature fusion module is able to maximize the MTS regression and classification performance by exploring the potential relationships between multiple scales from two directions.

[0017] S3. Input the training vectors at different scales obtained in step S1 into the multivariate time series signal analysis network for network training. In order to enable the network to be applied to classification and regression tasks in multivariate time series analysis, different loss functions are set during training. Among them, cross entropy loss is used for classification tasks, and MSE loss is used for regression tasks.

[0018] S4. After performing time-spectrum conversion and splicing on the multivariate time series signal to be processed in the manner of step S1, the signal is input into the trained multivariate time series signal analysis network obtained in step S3. For the classification task model, a classification result feature map of the signal is predicted; for the regression task model, a regression result feature map of the signal is predicted.

[0019] To validate the effectiveness of our method, experiments were conducted on the UEA archive and MIMIC III datasets. The UEA archive consists of 30 datasets, encompassing a wide range of cases, signal dimensions, and lengths across multiple domains. The cases cover fields such as medicine, human activity, audio, and meteorology; the signal dimensions range from (2,1345), the lengths from (8,17984), and the number of categories from (2,39). All signals in the UEA archive were used to evaluate the classification performance of multivariate time series signals. The MIMIC III data, collected from patients in the intensive care unit of a large tertiary care hospital, includes patient-related vital signs, laboratory measurements, and observations. In the MIMIC III dataset, photoplethysmography (PPG) and electrocardiography (ECG) signals were used to regress systolic blood pressure (SBP) and diastolic blood pressure (DBP) to evaluate the regression performance of multivariate time series signals.

[0020] The entire training and testing process was implemented using only one NVIDIA 1080Ti GPU with 11G memory.

[0021] On the UEA archive dataset, the proposed method was compared with nine state-of-the-art multivariate time series classification methods: Euclidean distance-based classifier (EDI), Dimension Independent Dynamic Time Warping (DTWI), Dimension-dependent Dynamic Time Warping (DTWD), Negative Samples (NS), Multivariate LSTM-FCNs (MLSTM), Multivariate Time Series Classification with Attentional Prototypical Network (TapNet), Shapelet-neural Network (ShapeNet), Omni-Scale CNNs (OS-CNN), and Dynamic Sparse Network (DSN). The proposed method was trained using the Adam optimizer with a learning rate of 1e-4, 50 training iterations, a learning rate of 1e-4, and a batch size of 64. To compare the classification performance of the different methods, the average accuracy (Average Accuracy) was used to measure their classification performance. The specific experimental results are shown in Table 1. As can be seen, the proposed method achieved a higher average classification accuracy.

[0022] Table 1

[0023]

[0024]

[0025] On the MIMIC III dataset, the proposed method was compared with eight state-of-the-art multivariate time series regression methods, including Cuffless Blood Pressure Estimation Algorithms (Kachuee), Multi-sensor Fusion Approach (Miao), Cuffless Deep Learning-based Blood Pressure Estimation (Song), Negative Samples (Sharifi), Dynamical Approach in Continuous Cuffless Blood Pressure Estimation (Miao), ViT, Hierarchical Vision Transformer Using Shifted Windows (Swin), and Shunted Self-Attention via Multi-Scale Token Aggregation (Shunted). The proposed method was trained using the Adam optimizer with a learning rate of 1e-4, 50 training iterations, a learning rate of 1e-4, and a batch size of 64. Mean absolute error (MAE), variance (SD), and absolute difference (ABSOLUTE_DIF) were used to evaluate the regression performance of the different methods. Table 2 shows the specific experimental results. It can be seen that the method of the present invention can achieve better regression performance.

[0026] Table 2

[0027]

[0028]

Claims

1. A multivariate time series signal analysis method based on multi-scale context, characterized by Here are the steps: S1. Given a multivariate time series signal training set, use short-time Fourier transform to convert the time series signal into a time-frequency spectrum, and splice the time-frequency spectrum of the same instance along the Z-axis direction. The spliced ​​time-frequency spectrum is divided into three scales of 8×8, 12×12, and 16×16 to obtain several slices of three scales; then, use linear mapping to map each slice into a one-dimensional vector of length 768, and use a vector of the same length to record the position information of the slice in the original signal as its position encoding vector; all slice vectors of the same scale are spliced ​​to obtain a multidimensional vector of length 768, which is used as the training parameter vector. The position encoding vector is then superimposed on the corresponding slice vector to obtain a network training vector at the same scale; S2. Construct a multivariate time series signal analysis network, including a multi-level encoding module and a bidirectional feature fusion module. The multi-level encoding module includes three hierarchical networks with the same structure and independent parameters. Each hierarchical network adopts the VisionTransformer architecture. The training vectors of three different scales are input into the three hierarchical networks respectively, and the feature maps at three scales are output. The bidirectional feature fusion module mainly includes a feature fusion module. The input of the feature fusion module is two different feature maps. The feature maps are first spliced. The spliced ​​feature maps pass through a convolution layer containing two convolution kernels with sizes of 1×1 and 3×3 respectively, and then pass through a convolution layer containing a convolution kernel with a size of 1×1. The fused feature maps are output by the sigmoid function. The specific processing process of the bidirectional feature fusion module is as follows: for the feature maps at three scales output by the multi-level encoding module, First, forward fusion is performed, that is, the two feature maps at the small scale and the medium scale are taken as input, and processed by the feature fusion module to obtain the fused feature map F1. Then, the feature map F1 and the feature map at the large scale output by the multi-level encoding module are taken as input, and processed by the feature fusion module to obtain the forward fused feature map F2; then, reverse fusion is performed, that is, the feature maps at the large scale and the medium scale output by the multi-level encoding module are taken as input, and processed by the feature fusion module to obtain the fused feature map F3; then, the feature map F3 and the feature map at the small scale output by the multi-level encoding module are taken as input, and processed by the feature fusion module to obtain the reverse fused feature map F4; finally, the feature map F2 obtained by forward fusion and the feature map F4 obtained by reverse fusion are taken as input, and processed by the feature fusion module to obtain the final fused feature map F5; S3. Input the training vectors at different scales obtained in step S1 into the multivariate time series signal analysis network for network training. If the cross entropy loss is used during training, a trained classification task model is obtained; if the MSE loss is used during training, a trained regression task model is obtained. S4. After performing time-spectrum conversion and splicing on the multivariate time series signal to be processed according to the method of step S1, the signal is input into the trained multivariate time series signal analysis network obtained in step S3. The output feature map is the final classification or regression result.

Images