A time series intelligent recognition method and system based on deep learning

Abstract

The present invention relates to a method and system for intelligent recognition of time series based on deep learning, the method comprising: constructing a TF‑DCRNN model based on CNN and RNN based on deep learning, combined with the STFT algorithm; constructing a time series data set and corresponding signal type labels for signal classification, the time series data set comprising a training set, a validation set and a test set; using the training set, the validation set and the test set, the TF‑DCRNN model is trained, validated and tested in turn, and the classification label is output; the classification label is compared with the signal type label to obtain the time series recognition result. The TF‑DCRNN model inherits the advantages of CNN and RNN, effectively reduces the amount of network hyperparameter training, and prevents overfitting; combines the time-frequency characteristics of the signal, increases the interpretability of the network, and improves the overall performance of the network.

Description

Technical Field

[0001] The present invention belongs to the technical field of time series intelligent recognition, and in particular, relates to a time series intelligent recognition method and system based on deep learning. Background Art

[0002] In recent years, with the rapid development of big data and artificial intelligence, deep learning methods based on feature learning and matching, such as convolutional neural network (CNN), recurrent neural network (RNN) and generative adversarial network (GAN), have been widely used in time series data recognition in the fields of natural earthquake monitoring, mechanical system fault detection, sound detection, etc.

[0003] Although deep learning methods such as CNN have advantages such as strong feature extraction capability and high efficiency, such methods are typical "black box models" and are unexplainable. It is difficult to find the logical basis of the CNN model for feature extraction and classification, which reduces the credibility of the results.

[0004] Most time domain signals are non-stationary signals with special time-frequency characteristics. Therefore, many researchers have incorporated time-frequency transformation methods into deep learning network models to increase interpretability and improve network performance. However, previous studies have mostly focused on data with high signal-to-noise ratios. Actual signals are often severely interfered with and have low data signal-to-noise ratios. Summary of the invention

[0005] In order to make up for the deficiencies of the prior art, the present invention provides a time series intelligent recognition method and system based on deep learning.

[0006] In order to solve the above technical problems, the technical solution adopted by the present invention is:

[0007] In a first aspect, a time series intelligent recognition method based on deep learning is provided, comprising:

[0008] Based on deep learning convolutional neural network CNN and recurrent neural network RNN, combined with short-time Fourier transform STFT algorithm, a time-frequency-depth convolutional recurrent network TF-DCRNN model is constructed;

[0009] Construct a time series dataset for signal classification and the corresponding signal type labels. The time series dataset includes a training set, a validation set, and a test set.

[0010] Use the training set, validation set, and test set to train, validate, and test the TF-DCRNN model in turn, and output classification labels;

[0011] The classification labels are compared with the signal type labels to obtain the time series recognition results.

[0012] Furthermore, the TF-DCRNN model includes:

[0013] Input layer, time-frequency convolution layer, standard convolution layer, pooling layer, flatten layer, long short-term memory layer, fully connected layer and output layer;

[0014] The input layer is used to input one-dimensional time series data;

[0015] The time-frequency convolution layer is connected to the input layer, and the time series data of the input layer is convolved using the STFT time-frequency analysis method to obtain two-dimensional data with time-frequency characteristics;

[0016] The standard convolution layer is a two-dimensional convolution kernel;

[0017] The pooling layer uses maximum pooling to extract features from two-dimensional data to obtain two-dimensional feature data;

[0018] The Flatten layer is used to flatten two-dimensional feature data into one-dimensional feature data;

[0019] The long short-term memory layer extracts long-term dependencies in time series by introducing gating mechanisms and long-term memory mechanisms;

[0020] The fully connected layer is used to convert the one-dimensional feature data mapping into classification labels;

[0021] The output layer is used to output classification labels.

[0022] Furthermore, the STFT time-frequency analysis method is used to perform time-frequency convolution on the time series data of the input layer to obtain two-dimensional data with time-frequency characteristics, including:

[0023] Using STFT time-frequency analysis method through the formula:

[0024] , for time series data Perform time-frequency convolution;

[0025] is the spectrum at time t, which is two-dimensional data with time-frequency characteristics; is the window function; is the triangular basis function.

[0026] Furthermore, a time series dataset for signal classification and corresponding signal type labels are constructed, including:

[0027] The Ricker wavelet is used to construct Ricker wavelet data sets under different signal-to-noise ratio conditions, and a pure random noise data set is constructed;

[0028] The Ricker wavelet data of at least two different frequency bands and a pure random noise data constitute a time series data set;

[0029] Corresponding signal type labels are set for Ricker wavelet data and pure random noise data of different frequency bands in the time series data set.

[0030] In the second aspect, a time series intelligent recognition system based on deep learning is provided, comprising:

[0031] Model building module, which is used to build the time-frequency-depth convolutional recurrent network TF-DCRNN model based on deep learning convolutional neural network CNN and recurrent neural network RNN, combined with the short-time Fourier transform STFT algorithm;

[0032] A data construction module is used to construct a time series data set for signal classification and the corresponding signal type label. The time series data set includes a training set, a validation set, and a test set.

[0033] The model processing module is used to train, verify and test the TF-DCRNN model in turn using the training set, validation set and test set, and output classification labels;

[0034] The time series recognition module is used to compare the classification label with the signal type label to obtain the time series recognition result.

[0035] Furthermore, the TF-DCRNN model includes:

[0036] Input layer, time-frequency convolution layer, standard convolution layer, pooling layer, flatten layer, long short-term memory layer, fully connected layer and output layer;

[0037] The input layer is used to input one-dimensional time series data;

[0038] The time-frequency convolution layer is connected to the input layer, and the time series data of the input layer is convolved using the STFT time-frequency analysis method to obtain two-dimensional data with time-frequency characteristics;

[0039] The standard convolution layer is a two-dimensional convolution kernel;

[0040] The pooling layer uses maximum pooling to extract features from two-dimensional data to obtain two-dimensional feature data;

[0041] The Flatten layer is used to flatten two-dimensional feature data into one-dimensional feature data;

[0042] The long short-term memory layer extracts long-term dependencies in time series by introducing gating mechanisms and long-term memory mechanisms;

[0043] The fully connected layer is used to convert the one-dimensional feature data mapping into classification labels;

[0044] The output layer is used to output classification labels.

[0045] Furthermore, the time-frequency convolution layer is used to use the STFT time-frequency analysis method through the formula:

[0046] , for time series data Perform time-frequency convolution;

[0047] is the spectrum at time t, which is two-dimensional data with time-frequency characteristics; is the window function; is the triangular basis function.

[0048] Furthermore, the data construction module is specifically used to use the Ricker wavelet to construct a Ricker wavelet data set under different signal-to-noise ratio conditions, and to construct a pure random noise data set; to form a time series data set with at least two Ricker wavelet data of different frequency bands and one pure random noise data; and to set corresponding signal type labels for the Ricker wavelet data and pure random noise data of different frequency bands in the time series data set.

[0049] The beneficial effects achieved by the present invention are:

[0050] The TF-DCRNN model was built by innovatively integrating the CNN and RNN network structures and the STFT algorithm. It inherits the advantages of CNN and RNN, can extract local features through the convolution layer, and use the long short-term memory layer for integration and memory, so that the convolution kernel has the ability to share weights, which can effectively reduce the amount of network hyperparameter training and prevent overfitting. At the same time, it can process data of any dimension and fully extract the multi-dimensional features of the data. It also combines the time-frequency characteristics of the signal to increase the interpretability of the network and improve the overall performance of the network. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] Figure 1 This is a flow chart of the time series intelligent recognition method based on deep learning of the present invention;

[0052] Figure 2 It is a comparison diagram of the convergence curve of the present invention;

[0053] Figure 3 This is a structural diagram of the time series intelligent recognition system based on deep learning of the present invention. DETAILED DESCRIPTION

[0054] The present invention will be further described below in conjunction with the accompanying drawings. The following embodiments are only used to more clearly illustrate the technical solution of the present invention, and cannot be used to limit the protection scope of the present invention.

[0055] like Figure 1 As shown, an embodiment of the present invention provides a time series intelligent recognition method based on deep learning, including:

[0056] 101, based on deep learning CNN and RNN, combined with STFT algorithm, to build TF-DCRNN model;

[0057] Convolutional neural network (CNN) and recurrent neural network (RNN) are two methods based on deep learning technology. CNN extracts features from input data by superimposing convolutional layers and pooling layers, and uses fully connected layers to achieve classification, prediction and other purposes. It performs well in image processing. The basic structure of RNN is a C-loop unit with memory mechanism and parameter sharing capabilities, which has great advantages in processing time series data.

[0058] In this embodiment, short time Fourier transform (STFT) is used as a time-frequency analysis method, CNN and RNN are combined with the STFT algorithm to construct a time-frequency deep convolutional recurrent neural network (TF-DCRNN) model. The TF-DCRNN model includes an input layer, a time-frequency convolution layer, a standard convolution layer, a pooling layer, a Flatten layer, a long short-term memory layer, a fully connected layer and an output layer.

[0059] The input layer is used to input one-dimensional time series data; the length of the time series data is not fixed, and multi-channel synchronous input is possible;

[0060] The time-frequency convolution layer is connected to the input layer, and the time series data of the input layer is convolved using the STFT time-frequency analysis method to obtain two-dimensional data with time-frequency characteristics; the STFT time-frequency analysis method is used through the formula: , for time series data Perform time-frequency convolution; is the spectrum at time t, which is two-dimensional data with time-frequency characteristics; is the window function; is the triangular basis function;

[0061] The standard convolution layer is a two-dimensional convolution kernel. Since a larger convolution kernel has a larger local receptive field, stacking multiple convolution kernels can improve the nonlinear processing capability. Therefore, the convolution kernel size is set to about 5×5 and has multiple scales. The number of convolution kernels in each layer is not less than 10.

[0062] The pooling layer uses maximum pooling to extract features from two-dimensional data to obtain two-dimensional feature data; the pooling layer can reduce the feature dimension and computational complexity. For non-steady-state signals, since the signal features account for a low proportion, it is more appropriate to use maximum pooling for feature extraction. At the same time, building multiple pooling layers can further extract signal features. Therefore, the number of maximum pooling layers should not be less than 2, and the size should be set to about 4×4.

[0063] The Flatten layer is used to flatten two-dimensional feature data into one-dimensional feature data;

[0064] The long short-term memory layer extracts long-term dependencies in the time series by introducing a gating mechanism and a long-term memory mechanism; the long short-term memory layer receives and outputs one-dimensional data; there can be multiple long short-term memory layers, but the more the number, the higher the computational complexity, and the long short-term memory layer should not be less than 2 layers;

[0065] The fully connected layer is used to convert the one-dimensional feature data mapping into classification labels; the fully connected layer introduces the fully connected operation at the last layer of the network;

[0066] The output layer is used to output classification labels; the number of classification categories depends on the actual situation, and each category corresponds to one label.

[0067] 102, constructing a time series data set for signal classification and a corresponding signal type label, the time series data set including a training set, a validation set, and a test set;

[0068] The training set is a dataset used to train the model, that is, to determine the model's learning parameters such as weights and biases; the validation set is used to select the model, more specifically, the validation set is used to select hyperparameters such as the number of network layers and learning rate; the test set is used to evaluate the model;

[0069] The Ricker wavelet data sets under different signal-to-noise ratio conditions are constructed by using the Ricker wavelet, and a pure random noise data set is constructed; the Ricker wavelet data of at least two different frequency bands and one pure random noise data form a time series data set; the Ricker wavelet data of different frequency bands and the pure random noise data in the time series data set are respectively set with corresponding signal type labels;

[0070] Assume that each time series data set contains 4 categories, of which 3 categories are Ricker wavelet data, with frequency bands of 1~60Hz, 61~150Hz and 151~250Hz, and labels corresponding to "1", "2" and "3" respectively; the fourth category is pure random noise data, with the corresponding label "0"; the total data set size is 80,000, and each label covers 20,000 data; in the categories labeled "1", "2" and "3", half are signals without noise interference, and the other half are signals interfered by random noise to varying degrees (a total of 4 tests were conducted, with signal-to-noise ratios (SNRs) of 0 dB, -5 dB, -10 dB and -15 dB respectively); in addition, the effective signal amplitude range is 0.1~1, and the amplitude range of the pure noise data set is 0~0.5. All data were normalized before training.

[0071] 103, using the training set, validation set and test set, train, validate and test the TF-DCRNN model in sequence, and output the classification label;

[0072] 104, compare the classification label with the signal type label to obtain the time series recognition result.

[0073] It should be noted that in order to evaluate the classification effect of the TF-DCRNN model, the following five comparative network models were constructed:

[0074] (1) CNN, a traditional one-dimensional CNN with 14 layers, including 2 convolutional layers and 3 maximum pooling layers. The number of convolution kernels in each convolutional layer is 10, and the convolution kernel sizes are 5×1 and 4×1 respectively;

[0075] (2) LSTM, with 11 layers, including 2 long short-term memory layers and 1 maximum pooling layer;

[0076] (3) CRNN, a combination of CNN and LSTM, with 18 layers, including 2 convolutional layers, 3 maximum pooling layers, and 2 long short-term memory layers;

[0077] (4) TF-CNN adds a time-frequency convolution layer to the traditional CNN. The network has 17 layers, including 2 convolution layers and 4 maximum pooling layers. Since the time-frequency convolution layer converts the one-dimensional time series data into a two-dimensional time-frequency feature map, the convolution kernels used in the subsequent convolution layers are two-dimensional, with sizes of 4×4 and 3×3 respectively, and the number of convolution kernels in each convolution layer is 10;

[0078] (5) TF-LSTM adds a time-frequency convolution layer to the traditional LSTM. The network has 12 layers, including 2 long short-term memory layers and 1 maximum pooling layer.

[0079] In all network models, the main common parameter settings are the same, as follows: the maximum number of cycles MaxEpochs is 30, the amount of data used in each iteration MiniBatchSize is 256, the initial learning rate InitialLearnRate is 0.001, and the optimization algorithm uses the random optimization of ADAM with high computational efficiency. In the network model calculation process, the data volume of the training set, validation set, and test set accounts for 60%, 20%, and 20% respectively;

[0080] The average value of the 50 experimental results is taken as the final classification result, and the average and variance comparison results of the classification accuracy of different network models are shown in Table 1 below;

[0081]

[0082] From the data in the table, we can see that CNN has better classification effect than LSTM. Combining the two or adding a time-frequency convolution layer can improve the network classification performance. Compared with other network models, TF-DCRNN has better classification effect, high classification accuracy and strong stability.

[0083] When the data signal-to-noise ratio is high (SNR≥-5), all network models have good classification effects (>90%). When the SNR is 0 dB and -5 dB, the accuracy of TF-DCRNN is 98.14% and 97.15%, respectively. As the data signal-to-noise ratio decreases, the classification effects of all network models tend to decline. When SNR=-10 dB, only CRNN and TF-DCRNN have classification accuracies greater than 90%. When the data is interfered by strong noise (SNR=-15 dB), the classification effects of traditional CNN and LSTM models are poor, with accuracies of 85.40% and 75.73%, respectively, and the network stability is reduced. The variance of LSTM is greater than 1, while the TF-DCRNN model can still maintain a good classification effect, with an accuracy of 90.26% and a variance less than 0.1.

[0084] like Figure 2 As shown in the figure, the convergence curve generated during a certain training process when SNR=-10 dB is displayed. It is obvious that the TF-DCRNN model has the best convergence effect, while the traditional CNN and LSTM have low convergence efficiency, and LSTM is prone to fall into local optimality. Embedding the time-frequency convolution layer into the traditional CNN or LSTM can improve the network performance to a certain extent, verifying the reliability of the TF-DCRNN model proposed in the present invention.

[0085] In combination with the time series intelligent recognition method based on deep learning described in the above embodiments, the time series intelligent recognition system based on deep learning is explained below through embodiments.

[0086] like Figure 3 As shown, a time series intelligent recognition system based on deep learning is provided, including:

[0087] The model building module 301 is used to build a time-frequency-depth convolutional recurrent network TF-DCRNN model based on the convolutional neural network CNN and the recurrent neural network RNN ​​of deep learning and combined with the short-time Fourier transform STFT algorithm;

[0088] A data construction module 302 is used to construct a time series data set for signal classification and a corresponding signal type label, where the time series data set includes a training set, a validation set, and a test set;

[0089] The model processing module 303 is used to train, verify and test the TF-DCRNN model in sequence using the training set, the validation set and the test set, and output the classification label;

[0090] The time series recognition module 304 is used to compare the classification label with the signal type label to obtain a time series recognition result.

[0091] Based on the description in the above embodiments of the time series intelligent recognition method and system based on deep learning, the present invention can bring the following beneficial effects:

[0092] The TF-DCRNN model was built by innovatively integrating the CNN and RNN network structures and the STFT algorithm. It inherits the advantages of CNN and RNN, can extract local features through the convolution layer, and use the long short-term memory layer for integration and memory, so that the convolution kernel has the ability to share weights, which can effectively reduce the amount of network hyperparameter training and prevent overfitting. At the same time, it can process data of any dimension and fully extract the multi-dimensional features of the data. It also combines the time-frequency characteristics of the signal to increase the interpretability of the network and improve the overall performance of the network.

[0093] It will be appreciated by those skilled in the art that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0094] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0095] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0096] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0097] The above are merely embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention are included in the scope of the claims of the present invention to be approved.

Claims

1. A time series intelligent recognition method based on deep learning, characterized in that: include: Based on deep learning convolutional neural network CNN and recurrent neural network RNN, combined with short-time Fourier transform STFT algorithm, a time-frequency-depth convolutional recurrent network TF-DCRNN model is constructed; Constructing a time series data set for signal classification and corresponding signal type labels, wherein the time series data set includes a training set, a validation set, and a test set; Using the training set, the validation set and the test set, the TF-DCRNN model is trained, validated and tested in sequence, and a classification label is output; Compare the classification label with the signal type label to obtain a time series recognition result; The TF-DCRNN model includes: Input layer, time-frequency convolution layer, standard convolution layer, pooling layer, flatten layer, long short-term memory layer, fully connected layer and output layer; The input layer is used to input one-dimensional time series data; The time-frequency convolution layer is connected to the input layer, and the time series data of the input layer are subjected to time-frequency convolution using the STFT time-frequency analysis method to obtain two-dimensional data with time-frequency characteristics; The standard convolution layer is a two-dimensional convolution kernel; The pooling layer uses maximum pooling to extract features from the two-dimensional data to obtain two-dimensional feature data; The Flatten layer is used to flatten the two-dimensional feature data into one-dimensional feature data; The long short-term memory layer extracts long-term dependencies in the time series by introducing a gating mechanism and a long-term memory mechanism; The fully connected layer is used to convert the one-dimensional feature data mapping into a classification label; The output layer is used to output the classification label; The method of performing time-frequency convolution on the time series data of the input layer by using the STFT time-frequency analysis method to obtain two-dimensional data with time-frequency characteristics includes: Using STFT time-frequency analysis method through the formula: Performing time-frequency convolution on the time series data h(τ); The STFT(t,ω) is the frequency spectrum at time t, which is two-dimensional data with time-frequency characteristics; the g(t) is the window function; the e -iωt is the triangular basis function; The construction of a time series data set for signal classification and corresponding signal type labels includes: The Ricker wavelet is used to construct Ricker wavelet data sets under different signal-to-noise ratio conditions, and a pure random noise data set is constructed; The Ricker wavelet data of at least two different frequency bands and a pure random noise data constitute a time series data set; Corresponding signal type labels are respectively set for the Ricker wavelet data and the pure random noise data of different frequency bands in the time series data set.

2. A time series intelligent recognition system based on deep learning, characterized in that: include: Model building module, which is used to build the time-frequency-depth convolutional recurrent network TF-DCRNN model based on deep learning convolutional neural network CNN and recurrent neural network RNN, combined with the short-time Fourier transform STFT algorithm; A data construction module, used to construct a time series data set for signal classification and a corresponding signal type label, wherein the time series data set includes a training set, a validation set, and a test set; A model processing module, used to train, verify and test the TF-DCRNN model in sequence using the training set, the validation set and the test set, and output a classification label; A time series recognition module, used for comparing the classification label with the signal type label to obtain a time series recognition result; The TF-DCRNN model includes: Input layer, time-frequency convolution layer, standard convolution layer, pooling layer, flatten layer, long short-term memory layer, fully connected layer and output layer; The input layer is used to input one-dimensional time series data; The time-frequency convolution layer is connected to the input layer, and the time series data of the input layer are subjected to time-frequency convolution using the STFT time-frequency analysis method to obtain two-dimensional data with time-frequency characteristics; The standard convolution layer is a two-dimensional convolution kernel; The pooling layer uses maximum pooling to extract features from the two-dimensional data to obtain two-dimensional feature data; The Flatten layer is used to flatten the two-dimensional feature data into one-dimensional feature data; The long short-term memory layer extracts long-term dependencies in the time series by introducing a gating mechanism and a long-term memory mechanism; The fully connected layer is used to convert the one-dimensional feature data mapping into a classification label; The output layer is used to output the classification label; The time-frequency convolution layer is used to use the STFT time-frequency analysis method through the formula: Performing time-frequency convolution on the time series data h(τ); The STFT(t,ω) is the frequency spectrum at time t, which is two-dimensional data with time-frequency characteristics; the g(t) is the window function; the e -iωt is the triangular basis function; The data construction module is specifically used to use Ricker wavelet to construct Ricker wavelet data sets under different signal-to-noise ratio conditions, and to construct a pure random noise data set; to form a time series data set with at least two Ricker wavelet data of different frequency bands and one pure random noise data; and to set corresponding signal type labels for the Ricker wavelet data and the pure random noise data of different frequency bands in the time series data set.

Images