Multivariate time series classification method and device integrating convolutional neural network and Transformer

Abstract

The present invention discloses a multivariate time series classification method that integrates convolutional neural networks and Transformer, and constructs a multivariate time series classification model. The multivariate time series classification model includes a multi-scale convolution module, a channel fusion convolution module, a feature dimension multi-head attention module, and a time dimension multi-head attention module. Fusion features are obtained through the multi-scale convolution module and the channel fusion convolution module. Feature time dimension abstract features are obtained based on the feature dimension multi-head attention module and the time dimension multi-head attention module, and prediction categories are further obtained. The present invention can simultaneously extract local and global features of time series, overcoming the limitations of traditional CNN and Transformer models when processing local or global features separately. Rich local features are extracted through multi-scale convolution operations, and the fusion of cross-channel information is enhanced through the channel fusion mechanism, thereby improving the expressiveness of the model in complex time series data.

Description

Technical Field

[0001] The present invention relates to the technical field of time series data processing, and more specifically to a multivariate time series classification method and device integrating a convolutional neural network and a Transformer. Background Art

[0002] Time series data is ubiquitous in real life, such as network traffic, electrocardiograms, sound data, and sensor data. Time series analysis studies this time-varying data and is widely used in fields such as activity recognition, electrocardiogram classification, and network anomaly detection. Based on the data's dimensionality, time series can be categorized as univariate and multivariate. In practical applications, multivariate time series are more common because they can simultaneously reflect the interrelationships between multiple variables and the complexity of system behavior. Time series classification (TSC) is a core task in time series analysis, aiming to classify data based on their changing patterns. Multivariate time series classification (MTSC) is more complex because it not only addresses the temporal characteristics of each variable but also considers the interdependencies between multiple variables. Because multivariate time series data often exhibit complex spatiotemporal dependencies, traditional classification methods often struggle to effectively capture this information. Therefore, research on MTSC is particularly important, particularly in areas such as smart healthcare, financial forecasting, and industrial monitoring, where it can provide more accurate data analysis and decision support, driving the development of various industries.

[0003] Existing methods for the Mobile Time Sequence (MTSC) task fall into two main categories: those based on traditional machine learning and those based on deep learning. The former relies on manual feature engineering and domain knowledge to improve classification performance through manually designed features, but is limited in effectiveness when dealing with high-dimensional, complex datasets and struggles to capture the data's deep structural features. The latter, on the other hand, uses neural networks to automatically learn multi-level feature representations from raw time series, reducing reliance on manual feature engineering and better adapting to complex and high-dimensional data. They particularly perform better in large-scale, complex data scenarios. Among deep learning-based MTSC methods, convolutional neural networks (CNNs) are widely used due to their ability to effectively extract local features from time series through local convolution operations. Models such as the Inception network, by using multi-scale convolution kernels in parallel, further improve understanding of complex patterns and classification performance. However, due to the limitations of convolution operations, CNN-based models lack the ability to capture global features. In contrast, the Transformer model, through its attention mechanism, effectively captures global features in time series, but its processing of temporal information (such as positional encoding) can weaken temporal dependencies in time series. Furthermore, existing Transformer models typically focus on extracting features from both the temporal and channel dimensions, making it difficult to efficiently capture both local and global features of time series. These limitations provide guidance for further optimizing model design in MTSC tasks. Summary of the Invention

[0004] The purpose of the present invention is to address the limitations of the existing technology in local feature extraction and global feature capture, and propose a multivariate time series classification method and device that integrates convolutional neural networks and transformers. The present invention first applies multi-scale two-dimensional convolution to each channel of the input time series sample through a multi-scale convolution module and a channel fusion convolution module, extracts local features using convolution kernels of different time steps, and simultaneously extracts features of each channel through a channel fusion convolution module and fuses cross-channel information to generate a high-dimensional feature representation containing multi-scale local features and channel information. Subsequently, the improved transformer model effectively retains time information by introducing learnable position encoding, and combines two multi-head attention mechanisms, namely feature dimension multi-head attention and time dimension multi-head attention, to capture the correlation between features and the dependency between time steps respectively. The present invention gives full play to the advantages of CNN in local feature extraction and the ability of Transformer in global feature capture, effectively making up for the shortcomings of a single model in processing complex time series data, and significantly improving the classification performance and the expression ability of the model.

[0005] In order to achieve the above object, the present invention adopts the following technical solutions:

[0006] The multivariate time series classification method that integrates convolutional neural networks and Transformer includes the following steps:

[0007] Step 1: Build a multivariate time series classification model. The multivariate time series classification model includes a multi-scale convolution module, a channel fusion convolution module, a feature dimension multi-head attention module, and a time dimension multi-head attention module.

[0008] Step 2: Collect the time series sample data Multi-scale convolution and channel fusion convolution operations are performed in turn through the multi-scale convolution module and the channel fusion convolution module to obtain fusion features ,

[0009] Step 3: Fusion features Remove the data type dimension and perform position encoding to obtain position encoding fusion features. The position encoding fusion features are input into the feature dimension multi-head attention module to obtain feature dimension residual connection features. , connect the feature dimension residual to the feature The channel dimension and time dimension of the exchange to obtain the residual connection feature , connect the residual feature Input into the time dimension multi-head attention module to obtain the time dimension residual connection feature , connect the time dimension residual feature After exchanging the time dimension and channel dimension, perform layer normalization and then input the feedforward neural network layer for nonlinear transformation and feature extraction, and apply layer normalization again to obtain the feature time dimension abstract features. ;

[0010] Step 4: Abstract features from the feature time dimension After average pooling of the time dimension, the time dimension is removed and then linear transformation and Normalize to obtain the predicted category of each time series sample;

[0011] Step 5: Train the multivariate time series classification model based on minimizing the loss function to obtain the optimal parameters of the multivariate time series classification model;

[0012] Step 6: Use the trained multivariate time series classification model to predict the time series sample set data to be identified.

[0013] As mentioned above, the multi-scale convolution and channel fusion convolution operations include the following steps:

[0014] Step 2.1: Assume that the time series sample set data is The dimensions of the time series sample set data include the total number of samples, the data type, and the time dimension. The time series sample set data X contains B time series samples, each of which contains D data types. Different data types correspond to different types of monitoring data of the monitoring object. Each data type includes T time point data. Add a channel dimension to the time series sample set data X to obtain the set data , represents the field of real numbers;

[0015] Step 2.2: Use scale convolution in the multi-scale convolution module kind, Represents the e-th scale convolution module For collection data The convolution operation results of each scale convolution module are spliced ​​together in the channel dimension to obtain the multi-scale local sensitive feature y, and then the multi-scale local sensitive feature After batch normalization, the GELU activation function is used to obtain enhanced multi-scale local sensitive features. ;

[0016] Step 2.3, channel fusion convolution module With Group convolution unit, channel fusion convolution module The size of the convolution kernel is , the number of output channels of the channel fusion convolution module is , the number of input channels is , is the number of output channels of the e-th scale convolution module, and then the channel fusion convolution module is used Sensitive to multi-scale local features Perform convolution operation to obtain fusion features , and then fusion features After batch normalization, GELU activation function is used to obtain fusion features .

[0017] As mentioned above, the e-th scale convolution module For collection data When performing convolution on the set data The time dimension is filled with 0 before and after, so that the time scale of the output features of each scale convolution module is Remain unchanged.

[0018] The position encoding fusion features in step 3 above are obtained based on the following steps:

[0019] Fusion features The data type dimension is compressed to obtain the fusion feature , set the learnable position encoding to , the position encoding copy Position code , the fusion features With positional encoding Add together to get the position encoding fusion feature .

[0020] As mentioned above, the feature dimension residual connection feature Obtained based on the following steps:

[0021] Fusion of positional encoding features Input the feature dimension multi-head attention module and calculate the first The query, key, and value of the attention head are further calculated. The calculation results of the attention head , the number of subspace dimensions , To round down, is the number of attention heads, and the calculation results of each attention head are Splice in the subspace dimension to obtain the splicing result , through the weight matrix The stitching results Perform linear transformation to obtain linear transformation result , and then the position encoding fusion feature And the linear transformation result Perform residual connection to obtain feature dimension residual connection features .

[0022] As mentioned above, the time dimension residual connection feature Obtained based on the following steps:

[0023] Connect the feature dimension residual to the feature The channel dimension and time dimension of are exchanged to obtain the residual connection feature , connect the residual feature Input to the time dimension multi-head attention module, calculate the first The query, key, and value of the attention head are further calculated. The calculation results of the attention head , the number of subspace dimensions , To round down, is the number of attention heads, and the calculation results of each attention head are Splice in the subspace dimension to obtain the splicing result , through the weight matrix The stitching results Perform linear transformation to obtain linear transformation result , and then connect the residual feature And the linear transformation result Perform residual connection to obtain time dimension residual connection features .

[0024] As mentioned above, the loss function is the cross entropy loss function.

[0025] A computer device includes a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the above classification method when executing the computer program.

[0026] A computer-readable storage medium stores a computer program, which implements the steps of the above classification method when executed by a processor.

[0027] A computer program product comprises a computer program, which implements the steps of the above classification method when executed by a processor.

[0028] Compared with the prior art, the present invention has the following advantages and effects:

[0029] 1. Through the multi-scale convolution module, channel fusion convolution module, feature dimension multi-head attention module and time dimension multi-head attention module, the present invention can simultaneously extract local and global features of time series, overcoming the limitations of traditional CNN and Transformer models when processing local or global features separately.

[0030] 2. The present invention extracts rich local features through multi-scale convolution operations and enhances the fusion of cross-channel information through the channel fusion mechanism, thereby improving the expressiveness of the model in complex time series data.

[0031] 3. Compared with traditional models, the present invention further improves the ability to understand time series data through the multi-head attention mechanism in the feature dimension and the multi-head attention mechanism in the time dimension, especially in the modeling of the dependency relationship between the time and feature dimensions, which better adapts to the multi-channel time series classification task and can effectively improve the classification accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] Figure 1 It is a structural diagram of the multivariate time series classification model (MSCFormer) of the present invention.

[0033] Figure 2 It is a visualization diagram of the multi-scale feature channel fusion module (multi-scale convolution module and channel fusion convolution module). Specific implementation plan

[0034] In order to facilitate those skilled in the art to understand and implement the present invention, the present invention is further described in detail below in conjunction with embodiments. It should be understood that the embodiments described herein are only used to illustrate and explain the present invention and are not used to limit the present invention.

[0035] Example 1

[0036] Time series data refers to a series of data points recorded in chronological order. Each data point typically consists of a continuous numerical value that changes over time. A time series can be viewed as a sequence of multiple data points, where the total number of data points represents the length of the series, while each data point itself contains multiple numerical values, reflecting the values ​​of different variables or channels at that moment. The goal of time series classification is to enable computers to automatically identify and determine the category to which a time series data point belongs. To achieve this goal, it is typically necessary to prepare a dataset containing a large amount of labeled time series data and use this data through a classification algorithm to train a model that can correctly classify new input time series data. Specifically, such a dataset contains multiple time series data points and their corresponding class labels. Each series has a label indicating its category. The number of records in the dataset represents the total number of time series data points, while these class labels come from a predefined set of categories. The core of multivariate time series classification is to accurately associate each input time series data point with its true category.

[0037] The basic framework of this method is shown in the attached Figure 1 shown.

[0038] Step 1: Construct a multivariate time series classification model (MSCFormer). The multivariate time series classification model (MSCFormer) includes a multi-scale convolution module, a channel fusion convolution module, a feature dimension multi-head attention module, and a time dimension multi-head attention module.

[0039] Step 2: Collect the time series sample data Multi-scale convolution and channel fusion convolution operations are performed in turn through the multi-scale convolution module and the channel fusion convolution module to obtain fusion features .

[0040] The purpose of this step is to improve feature extraction capabilities for time series data through multi-scale convolution and channel-fusion convolution operations. Multi-scale convolution uses convolution kernels of different scales to extract local features at multiple time scales, capturing both long-term and short-term patterns of variation and complex local dependencies in time series data. Next, channel-fusion convolution effectively fuses features from different channels, reducing redundant information and enhancing feature diversity and high-dimensional representation.

[0041] Step 2.1, suppose the time series sample set data is The dimensions of the time series sample set data include the total number of samples dimension (B), data type dimension (D), and time dimension (T). The time series sample set data X contains B time series samples, where B corresponds to the number of different monitoring objects. Each time series sample contains D data types, and different data types correspond to different types of monitoring data of the monitoring object. Each data type includes T time point data. Let the i-th time series sample be , Represents the real number domain, B represents the number of input time series samples, D is the data type number of the time point data, and T is the total number of time point data. Add a channel dimension to the time series sample set data X to obtain the set data Provide feature extraction for subsequent steps.

[0042] In this embodiment, the time series samples are time point data detected by the monitoring device worn by the monitored person at various time points in a certain activity time period. The number of time series samples is B, that is, the total number of monitored persons is B; the number of time point data is T, that is, the total number of time points is T; the number of data types D of the time point data is 6, that is, the time point data includes three-dimensional accelerometer data (3 dimensions) and three-dimensional gyroscope data (3 dimensions).

[0043] Step 2.2: Use scale convolution in the multi-scale convolution module species, respectively, are recorded as the first scale convolution module , the second scale convolution module ,…,E-th scale convolution module , the first scale convolution module , the second scale convolution module ,…,E-th scale convolution module The scales are the first scale , the second scale ,…,Eth scale , the e-th scale convolution module is recorded as , e∈{1,2,…,E}, where the e-th scale convolution module The size of the convolution kernel is , the number of output channels of the e-th scale convolution module is , the number of input channels is 1. Use the first scale convolution module , the second scale convolution module ,…,E-th scale convolution module For the collection data The convolution operations are:

[0044] (1)

[0045] in, Represents the convolution operation, the input channel is , the output channel is , Represents filling during convolution operation, that is, by The time dimension (the last dimension) is filled with 0 before and after to ensure that the input collection data And the time scale of the output features of each scale convolution module remain unchanged, Represents the e-th scale convolution module For collection data The convolution operation result is then Splice them together in the channel dimension (the second dimension) to obtain the multi-scale local sensitive feature y:

[0046] (2)

[0047] in, is the splicing function, and the obtained It is a set of multi-scale local sensitive features, and then the multi-scale local sensitive features After batch normalization (BN), the GELU activation function is used to further improve the stability and nonlinear expression ability of the model, and the enhanced multi-scale local sensitive features are obtained. .

[0048] Step 2.3, channel fusion convolution module With Group convolution unit, channel fusion convolution module The size of the convolution kernel is , the number of output channels of the channel fusion convolution module is , the number of input channels is , and then use the channel fusion convolution module Sensitive to multi-scale local features Perform convolution operations to effectively fuse information from different channels:

[0049] (3)

[0050] in, represents the channel fusion convolution module, the input channel is , the output channel is . Represents the fusion features output by the channel fusion convolution module. After batch normalization, the GELU activation function is used to further improve the stability and nonlinear expression ability of the model. The fusion features output by the enhanced fusion convolution module The operation in step 2 not only extracts deep features from each channel, but also captures the interaction information between channels, thereby generating a more comprehensive and high-dimensional feature representation. This approach helps reduce redundant information between features and improves the model's generalization and prediction performance. Figure 2 The change process of step 2 when B=1, E=2, M=2, and N=3 is visualized.

[0051] Step 3: Use the improved transformer model to enhance features. The improved transformer model includes a feature dimension multi-head attention module and a time dimension multi-head attention module. Specifically, it includes: Remove the data type dimension and perform position encoding to obtain position encoding fusion features. The position encoding fusion features are input into the feature dimension multi-head attention module to obtain feature dimension residual connection features. , connect the feature dimension residual to the feature The channel dimension and time dimension of the exchange to obtain the residual connection feature , connect the residual feature Input into the time dimension multi-head attention module to obtain the time dimension residual connection feature , connect the time dimension residual feature After exchanging the time dimension and channel dimension, perform layer normalization and then input the feedforward neural network layer for nonlinear transformation and feature extraction, and apply layer normalization again to obtain the feature time dimension abstract features. ;

[0052] The purpose of this step is to improve the model's ability to capture temporal dependencies in time series through the improved Transformer module. The learnable position encoding can dynamically adjust the position information between time steps, thereby enhancing sensitivity to temporal dependencies and improving prediction performance. The feature dimension multi-head attention module and the time dimension multi-head attention module promote information interaction between features and time steps respectively, capturing complex temporal dependencies. These operations can better understand the spatiotemporal patterns of time series and improve the expressiveness and robustness of feature representation. The specific process is:

[0053] Step 3.1: Fusion features The data type dimension (the third dimension) is squeezed to obtain the fusion feature , set the learnable position encoding to , where position encoding is a learnable parameter, position encoding is the learnable parameter of the Transformer module, and then the position encoding copy Position code , the fusion features With positional encoding Adding, the feature is integrated with the position coding information to obtain the position coding fusion feature .

[0054] Step 3.2: Fusion feature of position encoding in step 3.1 As input features, it is input into the feature dimension multi-head attention module, where For the number of output channels of the channel fusion convolution module in step 2.3, let:

[0055] (4)

[0056] in, They are the query, key and value of the feature dimension multi-head attention module respectively. The feature dimension multi-head attention module is set to have Attention heads map queries, keys, and values ​​into different subspaces:

[0057] (5)

[0058] in, represents the sequence number of the attention head, They are The learnable linear transformation matrix of the attention heads, 、 ,as well as Respectively The query, key, and value of the attention head, and the number of subspace dimensions , To round down, the attention operation for each subspace is:

[0059] (6)

[0060] in, represents the matrix transpose operator, Represents the subspace dimension corresponding to the attention head in the feature dimension multi-head attention module. represents the normalized exponential function, For the The calculation results of each attention head are calculated in the subspace dimension ( Dimensions) are spliced ​​together:

[0061] (7)

[0062] in, is the splicing function, This is the result of the splicing of the feature dimension multi-head attention module. Next, through the weight matrix The stitching results Perform linear transformation to obtain linear transformation result , and then the input features of the feature dimension multi-head attention module and the linear transformation results Perform residual connection to obtain feature dimension residual connection features , to prevent gradient explosion and increase training stability. Through this operation, high-dimensional features from different channels can interact with each other, generating richer and more comprehensive feature expressions, while enhancing the model's overall understanding of time series data.

[0063] Step 3.3: Connect the feature dimension residual to the feature The channel dimension (second dimension) and time dimension (third dimension) of are exchanged to obtain the residual connection feature , connect the residual feature As the input feature of the time dimension multi-head attention module, calculate the query of the time dimension multi-head attention module ,key Sum ,

[0064] (8)

[0065] in, They are the query, key and value of the time dimension multi-head attention module respectively. The time dimension multi-head attention module is set to have Attention heads map queries, keys, and values ​​into different subspaces:

[0066] (9)

[0067] in, represents the sequence number of the attention head, They are The learnable linear transformation matrix of the attention heads, 、 ,as well as Respectively The query, key, and value of the attention head, and the number of subspace dimensions , To round down, the attention operation for each subspace is:

[0068] (10)

[0069] in, represents the matrix transpose operator, Represents the subspace dimension corresponding to the attention head in the multi-head attention module in the time dimension. represents the normalized exponential function, For the The calculation results of each attention head are calculated in the subspace dimension ( Dimensions) are spliced ​​together:

[0070] (11)

[0071] in, is the splicing function, This is the result of splicing the multi-head attention module in the time dimension. Next, through the weight matrix The stitching results Perform linear transformation to obtain linear transformation result , and then the input features of the time dimension multi-head attention module (residual connection features ) and linear transformation results Perform residual connection to obtain time dimension residual connection features ,

[0072] The attention mechanism at this stage can identify and understand the mutual influence between each time step in high-dimensional features, thereby more accurately modeling time dependencies and improving module performance.

[0073] Step 3.4, then connect the time dimension residual feature Swap the time dimension (second dimension) and the channel dimension (third dimension) to obtain the time dimension residual connection feature , then the time dimension residual connection features Layer Normalization (LN) is applied to stabilize the training process and promote convergence, followed by nonlinear transformation and feature extraction through the Feed-Forward Network (FFN) layer, and layer normalization is applied again to obtain the feature time dimension abstract features. The feedforward neural network layer further improves the ability of feature representation, ensuring that the features processed by the attention mechanism have better interpretability and robustness.

[0074] Step 4: Abstract features from the feature time dimension Average pooling is performed on the time dimension to obtain features , the features The time dimension is removed to obtain the final feature , through the weight matrix For the final feature Perform linear transformation to obtain linear transformation features , for linear transformation features Implementation Normalize and obtain the predicted category of each time series sample.

[0075] The purpose of this step is to extract key information from the high-dimensional features after model feature extraction and use this information for the final classification task.

[0076] Step 4.1: First input the feature time dimension abstract feature of step 3.4 . Abstract features of feature time dimension The time dimension (the third dimension) is average pooled to obtain the features This step compresses the information of the time dimension and retains important global features. The time dimension is removed (squeeze) to obtain the final feature , at this time, the time series features of each time series sample are represented as an N-dimensional vector to input the fully connected layer. Next, the weight matrix For the final feature Perform linear transformation to obtain linear transformation features This step maps the N-dimensional feature space to the G-dimensional classification space, where G is the total number of categories in the classification space, and generates the original score for each classification category. Implementation Normalize and get Time series samples are classified into categories The predicted probability ,satisfy .remember For the The probability distribution of time series samples in each category, where the category corresponding to the maximum probability distribution is The final predicted category of the time series samples.

[0077] Step 5: When training the multivariate time series classification model (MSCFormer), a loss function is used to measure the difference between the model's predictions and the true labels, thereby guiding the model's optimization process. To ensure that the model can effectively classify multivariate time series, this step selects the cross-entropy loss function as the loss function. The multivariate time series classification model (MSCFormer) is trained based on minimizing the loss function to obtain the optimal parameters for the model (MSCFormer).

[0078] Step 6: After completing the training of the multivariate time series classification model (MSCFormer), you need to use the trained multivariate time series classification model (MSCFormer) to predict the time series sample set data (multivariate time series data) to be identified in order to evaluate its classification performance.

[0079] During the prediction phase, a batch of time series data to be classified must be prepared as time series samples, and a set of time series sample data to be identified must be constructed. Next, the trained multivariate time series classification model is loaded and placed in evaluation mode to avoid gradient updates or parameter changes during inference. Subsequently, the set of time series sample data to be identified is fed into the multivariate time series classification model. Upon receiving the set of time series sample data, the model performs forward propagation based on the multi-scale feature representations and temporal dependencies learned during training, extracting features layer by layer and performing calculations. Ultimately, it outputs the class probability distribution for each time series sample in the set of time series sample data to be identified. For each time series sample, the class with the highest probability is selected as the final prediction result, thereby determining the final predicted class for that time series sample. Finally, the entire batch of time series samples is aggregated to obtain the final predicted class for all time series samples.

[0080] In summary, our method (MSCFormer) extracts multi-scale local features and channel information from input data through multi-scale convolution and channel-fused convolution operations. It then leverages multi-head attention modules in both the feature dimension and the time dimension for global feature modeling. The synergistic effect of these steps makes the model more adaptable to time series data, ultimately achieving efficient classification of multi-channel time series. This combined approach fully leverages the strengths of both CNNs and Transformers, successfully overcoming the limitations of single models in feature extraction and modeling.

[0081] Those skilled in the art will understand that all or part of the processes in the above-mentioned embodiment methods can be implemented by instructing related hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods.

[0082] The advantages of the method of the present invention are verified by experiments below:

[0083] 1. Data Description

[0084] To test the effectiveness of this invention, we applied it to a time series data classification experiment. The time series datasets used in the experiment are all UEA time series classification datasets, which are widely used datasets for multivariate time series classification. The datasets used in the experiment are shown in Table 1.

[0085] Table 1 Dataset details

[0086]

[0087] 2. Description of Benchmark Methodology

[0088] To verify the superiority of the present invention, the present invention (MSCFormer model) is compared with the following seven advanced models, which are briefly introduced below:

[0089] 1. FEAT: A General Framework for Multivariate Time Series Representation Learning.

[0090] 2.DKN: A Transformer-based Dense Knowledge-aware Network for MTSC.

[0091] 3. MICOS: A hybrid supervised contrastive learning model for MSTC.

[0092] 4. ShapeNet: A novel neural network structure.

[0093] 5.Conv-GLU: Convolutional network with gated linear unit kernel.

[0094] 6.DA-NET: A neural network with dual attention.

[0095] 7.ConvTran: Transformer model based on improved positional encoding.

[0096] 3. Experimental Setup

[0097] There are 5 hyperparameters involved in the experiment, namely:

[0098] 1. Learning rate;

[0099] 2. The experiment uses 4 different scale combinations of convolution, the different scales are , and the corresponding number of output channels ;

[0100] 3. Number of output channels of channel fusion convolution kernel ;

[0101] 4. Number of heads in the multi-head attention mechanism in feature dimensions , the number of heads of the multi-head attention mechanism in the time dimension ;

[0102] According to the results of existing research, the number of heads of multi-head attention in the experiment and is directly set to 8. The number of output channels of each convolution kernel is set to the number of output channels of the channel fusion convolution kernel, that is, , the learning rate is set to 0.001, and the optimal values ​​of the remaining parameters are searched by grid search method. Their search ranges are: different scale combinations Search scope is ; Number of output channels of channel fusion convolution kernel The search scope is .

[0103] 4. Results of Algorithm Performance Comparison

[0104] Using the overall classification accuracy as the evaluation indicator, the experimental results are shown in Table 2 below:

[0105] Table 2 Comparison results

[0106]

[0107] The comparative experimental results, shown in Table 2, show that MSCFormer of the present invention outperforms all baseline models in terms of classification performance overall. Among the 27 datasets, MSCFormer of the present invention performs best on 15 datasets, significantly ahead of the baseline models, with an average accuracy of 80%, 4.1% higher than the best-performing baseline model. MSCFormer of the present invention performs very well on most datasets. For example, on datasets such as AWR, AF, CT, ER, and BM, MSCFormer of the present invention achieves or exceeds the highest accuracy. Even on some poorly performing datasets, MSCFormer of the present invention surpasses most baseline methods, further demonstrating its significant advantages in handling traditional time series tasks. In summary, MSCFormer of the present invention outperforms existing baseline models on multiple datasets, demonstrating its powerful ability to handle complex time series data.

[0108] Example 2

[0109] This embodiment provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the above-mentioned method embodiments when executing the computer program.

[0110] Example 3

[0111] This embodiment provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the steps of the above-mentioned method embodiments are implemented.

[0112] Example 4

[0113] This embodiment provides a computer program product, including a computer program. When the computer program is executed by a processor, the steps of the above method embodiments are implemented.

[0114] The specific embodiments described herein are merely illustrative of the spirit of the present invention. Those skilled in the art may modify, supplement, or replace the described specific embodiments with similar methods without departing from the spirit of the present invention or exceeding the scope defined by the appended claims.

Claims

1. The multivariate time series classification method that integrates convolutional neural networks and Transformer includes the following steps: Step 1: Build a multivariate time series classification model. The multivariate time series classification model includes a multi-scale convolution module, a channel fusion convolution module, a feature dimension multi-head attention module, and a time dimension multi-head attention module. Step 2: Collect the time series sample data Multi-scale convolution and channel fusion convolution operations are performed in turn through the multi-scale convolution module and the channel fusion convolution module to obtain fusion features , time series sample set data The time series samples in are the time point data detected by the monitoring device worn by the monitored person at various time points during a certain activity period. The time point data includes three-dimensional accelerometer data and three-dimensional gyroscope data. Step 3: Fusion features Remove the data type dimension and perform position encoding to obtain position encoding fusion features. The position encoding fusion features are input into the feature dimension multi-head attention module to obtain feature dimension residual connection features. , connect the feature dimension residual to the feature The channel dimension and time dimension of the exchange to obtain the residual connection feature , connect the residual feature Input into the time dimension multi-head attention module to obtain the time dimension residual connection feature , connect the time dimension residual feature After exchanging the time dimension and channel dimension, perform layer normalization and then input the feedforward neural network layer for nonlinear transformation and feature extraction, and apply layer normalization again to obtain the feature time dimension abstract features. ; Step 4: Abstract features from the feature time dimension After average pooling of the time dimension, the time dimension is removed and then linear transformation and Normalize to obtain the predicted category of each time series sample; Step 5: Train the multivariate time series classification model based on minimizing the loss function to obtain the optimal parameters of the multivariate time series classification model; Step 6: Use the trained multivariate time series classification model to predict the time series sample set data to be identified.

2. The multivariate time series classification method integrating convolutional neural network and Transformer according to claim 1, characterized in that: The multi-scale convolution and channel fusion convolution operations include the following steps: Step 2.1: Assume that the time series sample set data is The dimensions of the time series sample set data include the total number of samples, the data type, and the time dimension. The time series sample set data X contains B time series samples, each of which contains D data types. Different data types correspond to different types of monitoring data of the monitoring object. Each data type includes T time point data. Add a channel dimension to the time series sample set data X to obtain the set data , represents the field of real numbers; Step 2.2: Use scale convolution in the multi-scale convolution module kind, Represents the e-th scale convolution module For collection data The convolution operation results of each scale convolution module are spliced ​​together in the channel dimension to obtain the multi-scale local sensitive feature y, and then the multi-scale local sensitive feature After batch normalization, the GELU activation function is used to obtain enhanced multi-scale local sensitive features. ; Step 2.3, channel fusion convolution module With Group convolution unit, channel fusion convolution module The size of the convolution kernel is , the number of output channels of the channel fusion convolution module is , the number of input channels is , is the number of output channels of the e-th scale convolution module, and then the channel fusion convolution module is used Sensitive to multi-scale local features Perform convolution operation to obtain fusion features , and then fusion features After batch normalization, GELU activation function is used to obtain fusion features .

3. The multivariate time series classification method integrating convolutional neural network and Transformer according to claim 2, characterized in that: The e-th scale convolution module For collection data When performing convolution on the set data The time dimension is filled with 0 before and after, so that the time scale of the output features of each scale convolution module is Remain unchanged.

4. The multivariate time series classification method integrating convolutional neural network and Transformer according to claim 2, characterized in that: The position encoding fusion feature in step 3 is obtained based on the following steps: Fusion features The data type dimension is compressed to obtain the fusion feature , set the learnable position encoding to , the position encoding copy Position code , the fusion features With positional encoding Add together to get the position encoding fusion feature .

5. The multivariate time series classification method integrating convolutional neural network and Transformer according to claim 4, characterized in that: The feature dimension residual connection feature Obtained based on the following steps: Fusion of positional encoding features Input the feature dimension multi-head attention module and calculate the first The query, key, and value of the attention head are further calculated. The calculation results of the attention head , the number of subspace dimensions , To round down, is the number of attention heads, and the calculation results of each attention head are Splice in the subspace dimension to obtain the splicing result , through the weight matrix The stitching results Perform linear transformation to obtain linear transformation result , and then the position encoding fusion feature And the linear transformation result Perform residual connection to obtain feature dimension residual connection features .

6. The multivariate time series classification method integrating convolutional neural network and Transformer according to claim 5, characterized in that: The time dimension residual connection feature Obtained based on the following steps: Connect the feature dimension residual to the feature The channel dimension and time dimension of are exchanged to obtain the residual connection feature , connect the residual feature Input to the time dimension multi-head attention module, calculate the first The query, key, and value of the attention head are further calculated. The calculation results of the attention head , the number of subspace dimensions , To round down, is the number of attention heads, and the calculation results of each attention head are Splice in the subspace dimension to obtain the splicing result , through the weight matrix The stitching results Perform linear transformation to obtain linear transformation result , and then connect the residual feature And the linear transformation result Perform residual connection to obtain time dimension residual connection features .

7. The multivariate time series classification method integrating convolutional neural network and Transformer according to claim 1, characterized in that: The loss function is a cross entropy loss function.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the classification method according to any one of claims 1 to 7 are implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the classification method according to any one of claims 1 to 7 are implemented.

10. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the classification method according to any one of claims 1 to 7 are implemented.

Images