Single-variable time sequence classification method applying MHAGRU-MCCE

Through the MHAGRU-MCCE model, combining the multi-head attention mechanism and multi-scale conditional convolution enhancement module, the multi-scale features and time-dependent features of the time series are extracted, which solves the problem of insufficient classification accuracy in the existing technology and realizes significant classification Performance improvement.

CN119026027APending Publication Date: 2024-11-26SOUTH CHINA UNIV OF TECH
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411003009.8
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-25
Publication Date
2024-11-26

AI Technical Summary

Technical Problem

The existing univariate time series classification model has limitations in comprehensively utilizing multi-scale features and time-dependent features, resulting in insufficient classification accuracy.

Method used

Using the MHAGRU-MCCE model, combined with the GRU sequence processing branch optimized by multi-head attention mechanism and the multi-scale conditional convolution enhancement residual structural branch, the hybrid model is constructed to extract multi-scale features and time dependence through feature fusion and output block design. Features and train the model through cross entropy loss function.

Benefits of technology

It significantly improves the accuracy of time series classification and the model's understanding of time series data, improves classification performance, and has broad application prospects.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119026027A_ABST
    Figure CN119026027A_ABST
Patent Text Reader

Abstract

The invention discloses a univariate time sequence classification method applying MHAGRU-MCCE. The method comprises the following steps: acquiring a univariate time sequence data set; inputting the univariate time sequence into a residual structure branch, and extracting multi-scale features; inputting the univariate time sequence into a GRU sequence processing branch, and extracting a time-dependent feature; inputting the multi-scale features and the time-dependent features into a feature fusion and output block, outputting a classification result, and constructing an MHAGRU-MCCE hybrid model; and training the constructed MHAGRU-MCCE hybrid model, and evaluating the performance of the MHAGRU-MCCE hybrid model through the evaluation index. The method significantly improves the accuracy of univariate time sequence classification, is suitable for various univariate time sequence classification tasks, and has a wide application prospect.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the fields of deep learning, artificial intelligence technology, and univariate time series classification, and in particular to a univariate time series classification method using MHAGRU-MCCE (Multi-Head Attention mechanism based GRU-Multi-scale Conditional Convolution and Enhancement, multi-head attention gated recurrent unit and multi-scale conditional convolution enhancement model). Background Art

[0002] As a common data carrier and a collection of data points arranged in chronological order, time series is widely present in real-world fields such as industrial production, environmental monitoring, and medical health. Data mining is very important. Time series classification (TSC) is one of the core tasks. It accurately divides time series into predefined category labels based on the feature information analysis contained in the time series data. In the univariate TSC task, the effective use of the multi-scale features and time-dependent features of the time series is crucial to improving the classification accuracy. The existing models are limited in the comprehensive use of multi-scale features and time-dependent features.

[0003] This paper proposes a univariate time series classification method using MHAGRU-MCCE. This method can comprehensively extract and process multi-scale features and time-dependent features in time series, and significantly improve the accuracy of TSC through novel model structure design. In the field of univariate time series classification methods, there are similar patents such as CN114722990A, CN110119760B, CN113035361A, etc.:

[0004] The above-mentioned specific patent comparative documents are:

[0005] 1) "A time series classification method based on SAM-BiLSTM", patent number CN114722990A. The method discloses a time series classification method based on SAM-BiLSTM, including the following steps: obtaining a one-dimensional time series data set and normalizing the one-dimensional time series data set; constructing a time series classification network model based on SAM-BiLSTM; extracting features from the one-dimensional time series data to be tested through the trained SAM-BiLSTM time series classification model and outputting the time series classification result. The present invention combines the univariate time series input with the GRU sequence processing branch optimized by the multi-head attention mechanism MHA, which is composed of the multi-head attention mechanism MHA and GRU in series, to extract time-dependent features, which is different from the method of extracting time-dependent features using SAM-BiLSTM in the above invention.

[0006] 2) "A sequence classification method based on hierarchical multi-scale recurrent neural network", patent number CN110119760B. This method discloses a sequence classification method based on hierarchical multi-scale recurrent neural network, including the following steps: input a sequence and divide it into multiple sub-sequences of equal length; construct multiple pyramid structures in the order of sub-sequences, each pyramid receives a sub-sequence as input, generates a hidden state at the bottom of the pyramid and a hierarchical aggregation state at each level, and the aggregation state at the top of each pyramid is used as the input of the bottom of the next sub-pyramid; the aggregation states at the top of all pyramids are iteratively aggregated through jump connections to obtain the output of the layer; the hierarchical aggregation state sequences of different scales generated by all pyramids in the lower layer are used as the input of the higher layer to construct a multi-layer recurrent neural network to generate the output of each layer; the output of each layer is aggregated to obtain multi-scale fusion features; finally, a Softmax layer is used to classify the sequence based on the feature. The present invention inputs a univariate time series into a residual structure branch based on MCCE, which consists of a multi-scale conditional convolution enhanced residual block MCCE ResBolck, a maximum pooling layer, and a global average pooling layer. MCCE ResBolck is formed by stacking residual connections of two multi-scale conditional convolution enhanced modules MCCE to extract multi-scale features, which is different from the method of extracting multi-scale features using a pyramid structure in the above-mentioned invention.

[0007] 3) "A neural network time series classification method based on data enhancement", patent number CN113035361A. This method discloses a neural network time series classification method based on data enhancement. First, the time series data set is preprocessed; second, a suitable parameter α is selected to perform Mixup data enhancement on the training data, and the enhanced data is used for model training; then, an LSTM-FCN time series classification network model is constructed; finally, the cross entropy function is used as the loss function, and the LSTM-FCN network is trained using back propagation and gradient descent algorithm Adam. The MHAGRU-MCCE hybrid model disclosed in the present invention is composed of a residual structure branch based on a multi-scale conditional convolution enhancement module MCCE, a GRU sequence processing branch optimized by a multi-head attention mechanism MHA, and a feature fusion and output block part, which is different from the method of optimizing the LSTM-FCN model using Mixup data enhancement in the above invention. Summary of the invention

[0008] To solve the above technical problems, the purpose of the present invention is to provide a univariate time series classification method using MHAGRU-MCCE.

[0009] The purpose of the present invention is achieved through the following technical solutions:

[0010] A univariate time series classification method using MHAGRU-MCCE, comprising:

[0011] A. Obtain a univariate time series dataset;

[0012] B. Input the univariate time series into the residual structure branch to extract multi-scale features;

[0013] C. Input the univariate time series into the GRU sequence processing branch to extract time-dependent features;

[0014] D. Fusion of multi-scale features and time-dependent features into the output block, output of the classification results, and construction of the MHAGRU-MCCE hybrid model;

[0015] E. Train the constructed MHAGRU-MCCE hybrid model and evaluate the performance of the MHAGRU-MCCE hybrid model through evaluation indicators.

[0016] Compared with the prior art, one or more embodiments of the present invention may have the following advantages:

[0017] Through the residual structure branch based on the multi-scale conditional convolution enhancement module MCCE, the multi-scale features in the time series data can be effectively extracted, and the richness and accuracy of feature extraction can be improved; combined with the GRU sequence processing branch optimized by the multi-head attention mechanism MHA, the time-dependent features of the time series data can be fully captured, and the model's ability to understand the time series data can be improved; through the design of feature fusion and output blocks, the multi-scale features and time-dependent features are effectively fused, and the constructed MHAGRU-MCCE hybrid model has a significant improvement in classification performance; the method of the present invention can be applied to various univariate time series classification tasks and has broad application prospects. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 It is a flow chart of the univariate time series classification method using MHAGRU-MCCE;

[0019] Figure 2 It is the structural diagram of MHAGRU-MCCE model;

[0020] Figure 3 It is a multi-scale conditional convolution enhanced residual block MCCE ResBolck structure diagram;

[0021] Figure 4 It is the conditional convolution Condconv structure diagram;

[0022] Figure 5 This is the experimental result diagram of the MHAGRU-MCCE model on the UCR dataset. DETAILED DESCRIPTION

[0023] In order to make the purpose, technical solutions and advantages of the present invention more clear, the present invention will be further described in detail below in conjunction with embodiments and drawings.

[0024] like Figure 1 As shown in the figure, the univariate time series classification method process using MHAGRU-MCCE includes the following steps:

[0025] Step 10: Obtain a univariate time series data set;

[0026] Step 20: Input the univariate time series into the residual structure branch to extract multi-scale features;

[0027] Step 30: Input the univariate time series into the GRU sequence processing branch to extract the time-dependent features;

[0028] Step 40 integrates the multi-scale features and time-dependent features into the feature fusion and output block, outputs the classification result, and constructs the MHAGRU-MCCE hybrid model;

[0029] Step 50 trains the constructed MHAGRU-MCCE hybrid model and evaluates the performance of the MHAGRU-MCCE hybrid model through evaluation indicators.

[0030] The above step 10 specifically includes: obtaining a univariate time series data set, selecting 85 UCR univariate time series data sets in the field of univariate time series classification, and dividing the data set into a training set and a test set.

[0031] The above step 20 specifically includes: Figure 2 As shown, the residual structure branch is a residual structure branch based on a multi-scale conditional convolution enhancement module MCCE; the residual structure branch based on MCCE is composed of a multi-scale conditional convolution enhancement residual block MCCEResBolck, a maximum pooling layer, and a global average pooling layer, as shown in Figure 3 As shown in Figure 1, MCCE ResBolck is composed of two multi-scale conditional convolution enhancement modules MCCE stacked residual connections to extract multi-scale features.

[0032] The above step 20 specifically includes: the multi-scale conditional convolution enhancement module MCCE is formed by connecting a multi-scale conditional convolution block MCCB and an enhancement block EB in series.

[0033] The above step 20 specifically includes: the multi-scale conditional convolution block MCCB is composed of three independent parallel conditional convolution layers CondConv, a feature fusion layer, a batch normalization layer and a ReLU activation function, which are used to process and optimize the input features. The conditional convolution CondConv is the core block of the multi-scale conditional convolution block MCCB, such as Figure 4As shown in the figure, the conditional convolution CondConv first processes the input data x through the routing function r(x) composed of the maximum pooling GAP, the fully connected FC and the Sigmoid function, and then generates a dynamically adjusted weight parameter a i Control multiple groups of convolution kernels W i The dynamic combination is calculated as follows:

[0034]

[0035] The above step 20 specifically includes: the enhancement block EB is composed of a global average pooling layer GAP and two fully connected layers FC, which are used to further enhance and adjust the feature representation. The multi-scale conditional convolution block MCCB outputs the feature Y through GAP to obtain the global information of the time series data feature map The length of the feature map is L f , series characteristic element y n (i) The fused feature elements are expressed as The first layer FC uses the dimension reduction coefficient r to reduce the feature map dimension to the original input, and uses the ReLU activation function to add nonlinear processing. The second layer FC performs a dimensionality increase operation to restore it to the original dimension, and uses the Sigmoid function to calculate the weight S of each feature channel; the output weight S of the enhancement block EB is weighted channel by channel with the output feature Y of the multi-scale conditional convolution block MCCB to obtain the enhanced output Z. The calculation process is as follows:

[0036]

[0037] The above step 30 specifically includes: Figure 2 As shown, the GRU sequence processing branch is a GRU sequence processing branch optimized in combination with the multi-head attention mechanism MHA; it is composed of the multi-head attention mechanism MHA and GRU in series to extract time-dependent features.

[0038] The above step 40 specifically includes: the feature fusion and output block is composed of a feature fusion layer and a Softmax function; the feature fusion Concat is used to fuse the multi-scale features extracted from step 20 and the time-dependent features extracted from step 30, and the Softmax function is used to calculate the probability of each category and output the classification result, thereby completing the construction of the MHAGRU-MCCE hybrid model.

[0039] The above step 50 specifically includes: firstly, the constructed MHAGRU-MCCE hybrid model is trained using the cross entropy loss function, and the keras network framework equipped with NVIDIA GeForce RTX 3090 GPU is used. The specific parameter settings include: Max Pooling = 2, Dropout layer inactivation rate 0.8, and the number of convolution kernels of the three MCCE Resblocks is divided into 64, 128, and 256; the training is performed on the NVIDIA GeForce RTX 3090 GPU, and the optimizer is set to Adam, the initial learning rate = 0.001, the batch processing = 128, and the number of training rounds = 2000. The multi-scale conditional convolution block MCCB sets the convolution kernel sizes of 3 conditional convolutions CondConv = 3, 6, and 12, the convolution step size = 1, and the number of different convolution kernel groups = 2; the enhancement block EB sets the dimension reduction factor r = 16; MHAGRU sets the number of GRU memory cells = 8, 64, and 128, the number of MHA "heads" = 2, and uses the cross entropy loss function to optimize the model parameters. The model is trained and tested on 85 UCR univariate time series datasets. Finally, its performance is evaluated by the average accuracy MA and class average accuracy MPCA indicators. Figure 5 The experimental results of this method on the UCR dataset are shown in Figure 2. The calculation methods of MA and MPCA are as follows:

[0040]

[0041] In the formula, the total number of data sets is N, and the accuracy on the i-th data set is a i , the total number of categories is C, and the category accuracy is c m .

[0042] Although the embodiments disclosed in the present invention are as above, the above contents are only embodiments adopted for facilitating the understanding of the present invention and are not intended to limit the present invention. Any technician in the technical field to which the present invention belongs can make any modifications and changes in the form and details of the implementation without departing from the spirit and scope disclosed in the present invention, but the patent protection scope of the present invention shall still be subject to the scope defined in the attached claims.

Claims

1. A univariate time series classification method using MHAGRU-MCCE, It is characterized in that The method comprises: Step A: Get a univariate time series dataset Step B inputs the univariate time series into the residual structure branch to extract multi-scale features; Step C inputs the univariate time series into the GRU sequence processing branch to extract time-dependent features; Step D fuses the multi-scale features and time-dependent features into the output block, outputs the classification results, and constructs the MHAGRU-MCCE hybrid model; Step E trains the constructed MHAGRU-MCCE hybrid model and evaluates the performance of the MHAGRU-MCCE hybrid model through evaluation indicators.

2. The univariate time series classification method using MHAGRU-MCCE as claimed in claim 1, It is characterized in that In the step A, a univariate time series data set is obtained, and 85 UCR univariate time series data sets in the field of univariate time series classification are selected. The data set is divided into a training set and a test set.

3. The univariate time series classification method using MHAGRU-MCCE as claimed in claim 1, It is characterized in that In the step B, the residual structure branch is a residual structure branch based on a multi-scale conditional convolution enhancement module MCCE; the residual structure branch based on MCCE is composed of a multi-scale conditional convolution enhancement residual block MCCE ResBolck, a maximum pooling layer, and a global average pooling layer, wherein MCCE ResBolck is formed by stacking residual connections of two multi-scale conditional convolution enhancement modules MCCE.

4. The univariate time series classification method using MHAGRU-MCCE as claimed in claim 1, It is characterized in that In the step B, the multi-scale conditional convolution enhancement module MCCE is formed by connecting a multi-scale conditional convolution block MCCB and an enhancement block EB in series.

5. The univariate time series classification method using MHAGRU-MCCE as claimed in claim 4, It is characterized in that In the step B, the multi-scale conditional convolution block MCCB is composed of three independent parallel conditional convolution layers CondConv, a feature fusion layer, a batch normalization layer and a ReLU activation function, which is used to process and optimize the input features; The conditional convolution CondConv is the core block of the multi-scale conditional convolution block MCCB. The conditional convolution CondConv first processes the input data x through a routing function r(x) composed of maximum pooling GAP, full connection FC and Sigmoid function, and then generates a dynamically adjusted weight parameter a i Control multiple groups of convolution kernels W i The dynamic combination is calculated as follows:

6. The univariate time series classification method using MHAGRU-MCCE as claimed in claim 4, It is characterized in that In step B, the enhancement block EB is composed of a global average pooling layer GAP and two fully connected layers FC, which are used to further enhance and adjust the feature representation; the multi-scale conditional convolution block MCCB outputs the feature Y through GAP to obtain the global information of the time series data feature map The length of the feature map is L f , the characteristic element after the series characteristic element yn(i) is fused is expressed as The first layer FC uses the dimension reduction coefficient r to reduce the feature map dimension to the original input, and uses the ReLU activation function to add nonlinear processing. The second layer FC performs a dimensionality increase operation to restore it to the original dimension, and uses the Sigmoid function to calculate the weight S of each feature channel; the output weight S of the enhancement block EB is weighted channel by channel with the output feature Y of the multi-scale conditional convolution block MCCB to obtain the enhanced output Z. The calculation process is as follows:

7. The univariate time series classification method using MHAGRU-MCCE as claimed in claim 1, It is characterized in that In step C, the GRU sequence processing branch is a GRU sequence processing branch optimized in combination with a multi-head attention mechanism MHA, which is composed of a multi-head attention mechanism MHA and a GRU connected in series.

8. The univariate time series classification method using MHAGRU-MCCE as claimed in claim 1, It is characterized in that In the step D, the feature fusion and output block consists of a feature fusion layer and a Softmax function; the feature fusion Concat is used to fuse the extracted multi-scale features and the extracted time-dependent features, and the Softmax function is used to calculate the probability of each category and output the classification results, thereby completing the construction of the MHAGRU-MCCE hybrid model.

9. The univariate time series classification method using MHAGRU-MCCE as claimed in claim 1, It is characterized in that In the step E, the constructed MHAGRU-MCCE hybrid model is trained using the cross entropy loss function, and the performance of the MHAGRU-MCCE hybrid model is evaluated by the average accuracy MA and the category average accuracy MPCA indicators; MA and MPCA are calculated as follows: In the formula, the total number of data sets is N, and the accuracy on the i-th data set is a i , the total number of categories is C, and the category accuracy is c m .