Time sequence data time-frequency domain joint unified representation learning method based on spectral analysis

Abstract

The invention provides a time sequence data time-frequency domain joint unified representation learning method based on spectral analysis, and the method comprises the steps: inputting a coded time sequence into a time-frequency joint representation learning model for training, and obtaining time-frequency cross-domain unified representation of time sequence data outputted by the time-frequency joint representation learning model; obtaining the total loss of the time-frequency joint representation learning model based on the main loss and the order-preserving loss of the time-frequency joint representation learning model; updating structure parameters of the time-frequency joint representation learning model based on the total loss to obtain a trained time-frequency joint representation learning model; acquiring a time sequence corresponding to the application scene, and performing feature coding on the time sequence corresponding to the application scene to obtain an original time sequence corresponding to the application scene; and performing time-frequency cross-domain unified representation on the original time sequence corresponding to the application scene based on the trained time-frequency joint representation learning model to obtain time-frequency cross-domain unified representation of the time sequence data corresponding to the application scene.

Description

Technical Field

[0001] The present invention relates to the technical field of time series data analysis, and in particular to a time-frequency domain joint unified representation learning method for time series data based on spectrum analysis. Background Art

[0002] With the advent of the era of "Internet of Everything, Comprehensive Perception", various sensors are deployed and used in large quantities, causing the scale of time series data to explode. How to mine valuable information from massive time series data is becoming increasingly important. Therefore, time series data analysis research has attracted more and more people to participate.

[0003] In recent years, with the vigorous development of deep learning technology, with the help of the powerful hidden patterns and deep feature extraction capabilities of deep neural networks, time series data analysis methods based on deep learning, such as recurrent neural network-based methods, convolution-based methods, and Transformer methods based on self-attention mechanisms, have been proposed, and have achieved good results in tasks such as classification, anomaly detection, prediction, and missing value completion for time series data.

[0004] The above methods often focus on modeling the temporal changes and interaction patterns of time series data in pure time domain space, thereby completing different downstream tasks. However, time series data is often a highly abstract information body formed by the coupling of multiple components such as trend (sequence overall envelope), periodicity (multi-frequency domain components), and mutation (high-frequency components). It is extremely challenging to achieve a complete semantic representation of time series data by modeling only in the time domain space. Summary of the invention

[0005] The present invention provides a time-frequency domain joint unified representation learning method for time series data based on spectrum analysis, which can solve the above technical problems.

[0006] The present invention provides a time-frequency domain joint unified representation learning method for time series data based on spectrum analysis, the method comprising:

[0007] Perform feature encoding on the original time series to obtain the encoded time series;

[0008] The encoded time series is input into the time-frequency joint representation learning model for training, and the unified time-frequency cross-domain representation of the time series data output by the time-frequency joint representation learning model is obtained; wherein the time-frequency joint representation learning model includes L residually connected spectrum joint representation learning modules; during each training, the input data and output data of the current spectrum joint representation learning module are fused, and the fusion result is used as the input data of the next spectrum joint representation learning module; the input data of the first spectrum joint representation learning module is the encoded time series, and the fusion result of the output data and input data of the last spectrum joint representation learning module is the unified time-frequency cross-domain representation of the time series data output by the time-frequency joint representation learning model;

[0009] The main loss of the time-frequency joint representation learning model is obtained based on the unified time-frequency cross-domain representation of time series data and the labels corresponding to the unified time-frequency cross-domain representation of time series data;

[0010] Perform 1D convolution operation, linear transformation and normalization operation on the unified time-frequency cross-domain representation of the time series data to obtain the sorting information of the unified time-frequency cross-domain representation of the time series data;

[0011] The order-preserving loss of the time-frequency joint representation learning model is obtained based on the ordering information of the original time series and the ordering information of the time-frequency cross-domain unified representation of the time series data;

[0012] The total loss of the time-frequency joint representation learning model is obtained based on the main loss and order-preserving loss of the time-frequency joint representation learning model;

[0013] Based on the total loss, the structural parameters of the time-frequency joint representation learning model are updated to obtain an updated time-frequency joint representation learning model, and the original time series is continuously trained based on the updated time-frequency joint representation learning model until a preset number of training times is reached to obtain a trained time-frequency joint representation learning model;

[0014] Collecting the time series corresponding to the application scenario, performing feature encoding on the time series corresponding to the application scenario, and obtaining the original time series corresponding to the application scenario;

[0015] Based on the trained time-frequency joint representation learning model, the original time series corresponding to the application scenario is uniformly represented across time and frequency domains to obtain a unified time-frequency cross-domain representation of the time series data corresponding to the application scenario.

[0016] Preferably, the fusion result of the input data and output data of the current spectrum joint representation learning module is obtained by the following formula:

[0017]

[0018] In the formula, is the fusion result of the input data and output data of the lth spectral joint representation learning module, is the output data of the lth spectral joint representation learning module, is the input data of the lth spectral joint representation learning module.

[0019] Preferably, each spectrum joint representation learning module includes a time-frequency transformation module, a multi-scale parallel complex domain convolutional network and a feature fusion network connected in series;

[0020] The time-frequency transformation module is used to perform time-frequency transformation on the input data of the current spectrum joint representation learning module to obtain the data after time-frequency transformation;

[0021] A multi-scale parallel complex domain convolutional network is used to extract features from the data after time-frequency transformation to obtain feature-extracted data;

[0022] The feature fusion network is used to perform feature fusion on the data after feature extraction to obtain feature fused data, and the feature fused data is the output data of the current spectrum feature extraction and fusion module.

[0023] Preferably, the time-frequency transformation module is used to perform time-frequency transformation on the input data of the current spectrum joint representation learning module, and the data obtained after the time-frequency transformation includes:

[0024] Performing short-time Fourier transform on the input data of the current spectrum joint representation learning module to obtain several frequency components;

[0025] A parameter-free complex domain attention mechanism is used to adaptively screen several frequency components to obtain the time-frequency transformed data.

[0026] Preferably, the multi-scale parallel complex domain convolution network includes a plurality of complex domain convolution subnetworks connected in parallel and a splicing module, each complex domain convolution subnetwork includes two complex domain convolution modules connected in series, the convolution kernel size of each complex domain convolution module is 3×3, and the expansion rates of the complex domain convolution modules in different complex domain convolution subnetworks are different, which are 1, 3, 5, 7...;

[0027] Two complex domain convolution modules in each complex domain convolution subnetwork are used to extract multi-scale features from the time-frequency transformed data to obtain multi-scale features;

[0028] The splicing module is used to splice multi-scale features to obtain the data after feature extraction.

[0029] Preferably, the main loss is obtained by the following formula:

[0030] Loss_main=MSE(Y,Y label )

[0031] The order-preserving loss is obtained by the following formula:

[0032] Loss_series = MSE(X series ,Y series )

[0033] The total loss is obtained by the following formula:

[0034] Loss = Loss_main + Loss_series

[0035] In the formula, Loss_main is the main loss, MSE(Y,Y label ) is Y,Y label The root mean square error of Y is the unified representation of the time-frequency cross-domain of the time series data, Y label is the time-frequency cross-domain unified representation of the corresponding label of the time series data, Loss_series is the order-preserving loss, MSE(X series ,Y series ) is X series ,Y series The root mean square error, X series is the sorting information of the original time series, Y series It is the sorting information uniformly represented across time and frequency domains of time series data, and Loss is the total loss.

[0036] Preferably, the original time series is power data, financial data, weather data, medical data or traffic data.

[0037] The technical solution of the present invention is applied. First, a multi-dimensional semantic embedding fusion representation network is used to perform high-dimensional semantic fusion mapping on time series data. Then, the time-frequency analysis tool is used to convert the pure time domain data to the time-frequency domain. The data structure is converted from 1D to 2D space. Then, a parameter-free complex domain attention mechanism is used to adaptively screen each frequency component to reduce the information redundancy caused by too many frequency components and improve the role of key frequency components. Then, the learnable 2D multi-scale parallel complex domain convolution is used to perform joint extraction of time change patterns and frequency change patterns. Finally, deep semantic fusion is performed through the feature fusion module to construct a complete semantic representation structure for time series data. The time-frequency domain joint unified representation method for time series data based on spectral analysis described in the present invention breaks through the semantic representation bottleneck brought by 1D time series data, realizes efficient joint extraction of time-frequency features through 2D convolution kernels, and greatly improves the time series data representation capability. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] The included drawings are used to provide a further understanding of the embodiments of the present invention, which constitute a part of the specification, are used to illustrate the embodiments of the present invention, and together with the text description, explain the principles of the present invention. Obviously, the drawings in the following description are only some embodiments of the present invention, and for ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0039] Figure 1 A flowchart of a time-frequency domain joint unified representation learning method for time series data based on spectrum analysis according to an embodiment of the present invention is shown;

[0040] Figure 2 A time-frequency conversion flow chart provided according to an embodiment of the present invention is shown;

[0041] Figure 3 A diagram showing experimental results of a classification task provided according to an embodiment of the present invention is shown. DETAILED DESCRIPTION

[0042] It should be noted that, in the absence of conflict, the embodiments in this application and the features in the embodiments can be combined with each other. The technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. The following description of at least one exemplary embodiment is actually only illustrative and is by no means intended to limit the present invention and its application or use. Based on the embodiments in the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0043] It should be noted that the terms used herein are only for describing specific embodiments and are not intended to limit the exemplary embodiments according to the present application. As used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. In addition, it should be understood that when the terms "comprise" and / or "include" are used in this specification, it indicates the presence of features, steps, operations, devices, components and / or combinations thereof.

[0044] Unless otherwise specifically stated, the relative arrangement of the parts and steps described in these embodiments, numerical expressions and numerical values ​​do not limit the scope of the present invention. At the same time, it should be understood that, for ease of description, the sizes of the various parts shown in the accompanying drawings are not drawn according to the actual proportional relationship. The technology, method and equipment known to ordinary technicians in the relevant field may not be discussed in detail, but in appropriate cases, the technology, method and equipment should be regarded as a part of the authorization specification. In all examples shown and discussed here, any specific value should be interpreted as being merely exemplary, rather than as a limitation. Therefore, other examples of exemplary embodiments may have different values. It should be noted that similar reference numerals and letters represent similar items in the following drawings, so once a certain item is defined in an accompanying drawing, it does not need to be further discussed in subsequent drawings.

[0045] like Figure 1 As shown, the present invention provides a time-frequency domain joint unified representation learning method for time series data based on spectrum analysis

[0046] The present invention provides a time-frequency domain joint unified representation learning method for time series data based on spectrum analysis, the method comprising:

[0047] Perform feature encoding on the original time series to obtain the encoded time series;

[0048] The encoded time series is input into the time-frequency joint representation learning model for training, and the unified time-frequency cross-domain representation of the time series data output by the time-frequency joint representation learning model is obtained; wherein, the time-frequency joint representation learning model includes L residually connected spectrum joint representation learning modules (Spectrum Joint Representation Learning Block, SJRLBlock); in each training, the input data and output data of the current spectrum joint representation learning module are fused, and the fusion result is used as the input data of the next spectrum joint representation learning module; the input data of the first spectrum joint representation learning module is the encoded time series, and the fusion result of the output data and input data of the last spectrum joint representation learning module is the unified time-frequency cross-domain representation of the time series data output by the time-frequency joint representation learning model;

[0049] The main loss of the time-frequency joint representation learning model is obtained based on the unified time-frequency cross-domain representation of time series data and the labels corresponding to the unified time-frequency cross-domain representation of time series data;

[0050] Perform 1D convolution operation, linear transformation and normalization operation on the unified time-frequency cross-domain representation of the time series data to obtain the sorting information of the unified time-frequency cross-domain representation of the time series data;

[0051] The order-preserving loss of the time-frequency joint representation learning model is obtained based on the ordering information of the original time series and the ordering information of the time-frequency cross-domain unified representation of the time series data;

[0052] The total loss of the time-frequency joint representation learning model is obtained based on the main loss and order-preserving loss of the time-frequency joint representation learning model;

[0053] Based on the total loss, the structural parameters of the time-frequency joint representation learning model are updated to obtain an updated time-frequency joint representation learning model, and the original time series is continuously trained based on the updated time-frequency joint representation learning model until a preset number of training times is reached to obtain a trained time-frequency joint representation learning model;

[0054] Collecting the time series corresponding to the application scenario, performing feature encoding on the time series corresponding to the application scenario, and obtaining the original time series corresponding to the application scenario;

[0055] Based on the trained time-frequency joint representation learning model, the original time series corresponding to the application scenario is uniformly represented across time and frequency domains to obtain a unified time-frequency cross-domain representation of the time series data corresponding to the application scenario.

[0056] In the present invention, firstly, a multi-dimensional semantic embedding fusion representation network is used to perform high-dimensional semantic fusion mapping on time series data, and then the time-frequency analysis tool is used to convert the pure time domain data to the time-frequency domain, and the data structure is converted from 1D to 2D space. Then, a parameter-free complex domain attention mechanism is used to adaptively screen each frequency component, reduce the information redundancy caused by too many frequency components, and improve the role of key frequency components. Then, the learnable 2D multi-scale parallel complex domain convolution is used to perform joint extraction of time change patterns and frequency change patterns. Finally, deep semantic fusion is performed through the feature fusion module to construct a complete semantic representation structure of time series data. The time-frequency domain joint unified representation method of time series data based on spectral analysis described in the present invention breaks through the semantic representation bottleneck brought by 1D time series data, realizes efficient joint extraction of time-frequency features through 2D convolution kernels, and greatly improves the time series data representation capability.

[0057] For a time series X∈R with a time length of T and N variables T×N (It is particularly noted that the present invention uses X and S to represent 1D time series data and 2D time-frequency data obtained after time-frequency transformation, and R represents the real number domain). First, a multi-dimensional semantic embedding fusion representation network is used to perform high-dimensional mapping, and the calculation formula is as follows:

[0058] X 0 =Encf(X)

[0059] Among them, Encf(·) is a multi-dimensional semantic embedding fusion representation network, which consists of a position encoder, a linear semantic encoder, and a temporal information encoder. 0 ∈R T×M And M is the feature dimension after encoding.

[0060] The lth SJRLBlock processing result Input into the l+1th SJRLBlock, the processing process can be expressed as the following formula:

[0061]

[0062] In the formula, is the fusion result of the input data and output data of the lth spectral joint representation learning module, is the output data of the lth spectral joint representation learning module, is the input data of the lth spectral joint representation learning module.

[0063] According to an embodiment of the present invention, each spectrum joint representation learning module includes a time-frequency transformation module, a multi-scale parallel complex domain convolution network and a feature fusion network connected in series;

[0064] The time-frequency transformation module is used to perform time-frequency transformation on the input data of the current spectrum joint representation learning module to obtain the data after time-frequency transformation;

[0065] A multi-scale parallel complex domain convolutional network is used to extract features from the data after time-frequency transformation to obtain feature-extracted data;

[0066] The feature fusion network is used to perform feature fusion on the data after feature extraction to obtain feature fused data, and the feature fused data is the output data of the current spectrum feature extraction and fusion module.

[0067] Specifically, for any l-th SJRLBlock, the detailed data processing flow is as follows: first, the l-1-th SJRLBlock processing result Perform time-frequency transformation to realize the conversion of 1D time series data into 2D data space, and then use 2D convolution to perform multi-scale feature extraction and feature fusion. The process can be expressed as follows:

[0068]

[0069]

[0070] Among them, TFT(·) represents the time-frequency transformation module, (C represents the complex domain) is the result obtained after being processed by the time-frequency transformation module (F represents the frequency dimension), MPCCN(·) represents a multi-scale parallel complex domain convolutional neural network, is the feature extraction result obtained after processing by this module (G represents the number of feature maps of different scales obtained). FAG(·) represents the feature fusion network. It is the result of feature fusion.

[0071] like Figure 2 As shown, according to an embodiment of the present invention, a time-frequency transformation module is used to perform time-frequency transformation on the input data of the current spectrum joint representation learning module, and the data obtained after the time-frequency transformation includes:

[0072] Performing short-time Fourier transform on the input data of the current spectrum joint representation learning module to obtain several frequency components;

[0073] A parameter-free complex domain attention mechanism is used to adaptively screen several frequency components to obtain the time-frequency transformed data.

[0074] Specifically, in the time-frequency transformation module (TFT), short-time Fourier transform (STFT) is used to decouple the periodic characteristics of the time series data, transform the time series data from the time domain to the time-frequency domain, and realize the expansion of the time series data structure representation from 1D to 2D data space. Then, the non-parametric complex domain attention mechanism is used to screen the key frequency components, and finally a high-information-density 2D time series data representation is obtained. The time-frequency transformation process of time series data is as follows: Figure 2 shown.

[0075] Usually, most of the time series data in practical applications are non-stationary signals. Therefore, short-time Fourier transform can be used to obtain the spectrum. Then, the present invention uses the stft function in pytorch to perform short-time Fourier transform calculation on the time series data. The calculation formula is expressed as follows:

[0076]

[0077] Among them, STFT(·) represents short-time Fourier transform. Hop_length represents the overlapping step length between adjacent windows. The overlapping of adjacent windows is to prevent the loss of time information. Window_F represents the window function. The window function can reduce the spectrum leakage when the signal is intercepted during the time-frequency transformation process. According to different task requirements in practical applications, window_F in the present invention uses hanning window, hamming window and rectangular window. n_fft represents the fast Fourier calculation window length, win_length represents the window length of short-time Fourier transform. In the present invention, n_fft is consistent with win_length, and win_lengt∈[8,16,24]. is the result of short-time Fourier transform calculation.

[0078] After the time series data is transformed by short-time Fourier transform, multiple frequency components are obtained. Then, the present invention will use the non-parametric complex domain attention mechanism to adaptively screen each frequency component to reduce the information redundancy caused by too many frequency components and enhance the role of key frequency components. The calculation formula of the non-parametric complex domain attention mechanism is as follows:

[0079]

[0080] in, represents the data after time-frequency transformation, sigmoid(·) represents the activation function, K (key matrix), Q (query matrix), and V (value matrix) are all And the “·” dot multiplication operation follows the principle of complex number multiplication.

[0081] According to an embodiment of the present invention, a multi-scale parallel complex domain convolution network includes a plurality of complex domain convolution subnetworks connected in parallel and a splicing module, each complex domain convolution subnetwork includes two complex domain convolution modules connected in series, a convolution kernel size of each complex domain convolution module is 3×3, and expansion rates of complex domain convolution modules in different complex domain convolution subnetworks are different, namely, 1, 3, 5, 7...;

[0082] Two complex domain convolution modules in each complex domain convolution subnetwork are used to extract multi-scale features from the time-frequency transformed data to obtain multi-scale features;

[0083] The splicing module is used to splice multi-scale features to obtain the data after feature extraction.

[0084] Specifically, the time series data is processed by the above-mentioned time-frequency transformation module to obtain a complex domain matrix In order to meet the needs of complex convolution operations, the present invention performs traditional real-valued two-dimensional convolution operations in the complex domain. The specific calculation process is as follows:

[0085] Assume that for any complex convolution kernel weight parameter W = A 0 +j*B 0 , A 0 , B 0 is any real matrix of the same dimension, the complex input tensor matrix H = A 1 +j*B 1 , A 1 , B 1 For any real matrix of the same dimension, then the complex convolution process can be expressed as follows:

[0086] W*H=(A 0 +j*B 0 )*(A 1 +j*B 1 )=(A 0 A 1 -B 0 B 1 )+j*(A 0 B 1 +A 1 B 0 )

[0087] Based on the above complex domain convolution calculation process, the present invention designs a multi-scale parallel complex domain convolution neural network. The network parallelizes G complex domain convolution subnetworks ComplexConv2D_Block_i(1,2,...,g). Each complex domain convolution subnetwork contains two complex domain convolution operations, and the convolution kernel size in each complex domain convolution module is 3×3, with different expansion rates (using 1, 3, 5, 7...). Based on the same convolution kernel, multi-scale feature extraction is realized under different expansion rates. Then, the multi-scale features are aggregated to obtain the time-frequency features of the time series data. The calculation process is expressed as follows:

[0088]

[0089] The feature extraction network designed by the present invention adopts 2D convolution to more efficiently perform joint extraction of time-frequency features on the time-frequency series data after time-frequency transformation, while capturing multi-scale frequency change and time change characteristics. The multi-scale parallel convolution network designed by the dilated convolution method, on the one hand, can perform multi-scale modeling of the correlation between time series variables, which is conducive to extracting the long-distance dependency of time series data; on the other hand, by deeply mining the mutation pattern of the time series data represented by the high-frequency component in multiple continuous scale spaces, it can also effectively capture the transient characteristics of the time series data. In addition, the feature extraction network designed by the present invention adopts the complex domain convolution method, which is more suitable for the comprehensive and integrated use of complex frequency domain information.

[0090] According to an embodiment of the present invention, in the feature fusion network, the input Perform the stacking operation of the real and imaginary parts (Stack(·)), and obtain the unified time-frequency cross-domain representation of the time series data after the linear transformation (Lear(·)) and the mean operation (Mean(·)). The calculation formula is as follows:

[0091]

[0092] The loss function is mainly used to measure the quality of the model output results during the training process, thereby guiding the optimization of model parameters. In the setting of the loss function in this invention, a learning strategy guided by "sequence" prior information is adopted, so as to more efficiently capture the global change trend pattern of time series data and maximize the orderliness of the model output results. Specifically, the order-preserving loss function Loss_series is added to the loss function of the model, and the calculation process is as follows:

[0093] Y series =Linear(Con1D(Y))

[0094] Loss_series = MSE(X series ,Y series )

[0095] For Y∈R B×T (Y represents the model output calculation result, which is the unified representation of the time-frequency cross-domain of the time series data) After performing 1D convolution operation Con1D(·) and linear transformation Linear(·), the sorting information Y of the output result is obtained. series ∈R B×T , X series ∈R B×T is the arrangement information of the original time series, and then the mean square error MSE(·) between the two is calculated and recorded as the order-preserving loss value, where X series and Y series All normalization operations were performed.

[0096] Therefore, in the prediction task, the loss function consists of two parts: the main loss function Loss_main and the order-preserving loss function Loss_series. The calculation formula is as follows:

[0097] The main loss is obtained by the following formula:

[0098] Loss_main=MSE(Y,Y label )

[0099] The total loss is obtained by the following formula:

[0100] Loss = Loss_main + Loss_series

[0101] In the formula, Loss_main is the main loss, MSE(Y,Y label ) is Y,Y label The root mean square error of Y is the unified representation of the time-frequency cross-domain of the time series data, Y label is the time-frequency cross-domain unified representation of the corresponding label of the time series data, Loss_series is the order-preserving loss, MSE(X series ,Y series ) is X series ,Y series The root mean square error, X series is the sorting information of the original time series, Y series It is the sorting information uniformly represented across time and frequency domains of time series data, and Loss is the total loss.

[0102] According to an embodiment of the present invention, the original time series is power data, financial data, weather data, medical data or traffic data. The original time series belongs to the data in this article.

[0103] Specifically, power data may include power transformer temperature data or power consumption data, financial data may include daily exchange rate data, weather data may include temperature, humidity, etc., and traffic data may include road occupancy rates measured by different sensors.

[0104] In order to further understand the present invention, the following Figure 3 The time-frequency domain joint unified representation learning method for time series data based on spectrum analysis of the present invention is described in detail.

[0105] In order to verify the effectiveness and superiority of the time-frequency joint representation learning model proposed in this paper in time series data analysis, experiments will be conducted on five major tasks in the field of time series data analysis (classification, anomaly detection, prediction, and missing value completion). The benchmark datasets and related configuration item settings used in the experiment are shown in Table 1:

[0106] Table 1 Overview of data sets, measurement indicators and sequence lengths

[0107]

[0108]

[0109] In addition, the comparison algorithms used in this experiment include the most advanced RNN models (LSTNet, LSSL), CNN models (TimesNet, TCN), MLP models (LightTS, Dlinear) and Transformer models (FEDformer, Informer, Pyraformer, Stationary, Informer, ETSformer).

[0110] In addition, the algorithm evaluation index settings are used in the verification experiment: the accuracy is used in the classification task; the F1 value is used in the outlier detection task, which is the harmonic mean of precision and recall. The root mean square error MSE and the mean absolute error MAE are used in the long-term and short-term prediction tasks and the missing value completion task. In the short-term prediction task, referring to N-BEATS, the mean absolute percentage error (SMAPE), the mean absolute proportion error (MASE) and the overall weighted average (OWA) are used, among which OWA is the metric used in the M4 competition.

[0111] It should be noted that the evaluation index used in the classification task is the average accuracy; the evaluation index used in the anomaly detection task is the average F1 value; the evaluation index used in the short-term prediction task is OWA; the evaluation index used in the long-term prediction task is the average MSE under different prediction lengths; the evaluation index used in the missing value completion task is the average MSE under different missing value conditions. The underlined in the experimental results table indicates the best.

[0112] (1) Classification task

[0113] In this experiment, the data used is from the UAE time series dataset, which includes 10 sub-datasets, covering practical scenario tasks such as gesture recognition, action recognition, audio recognition, and medical diagnosis. The experimental results are as follows: Figure 3 As shown, it can be seen that the method proposed in the present invention has achieved the best results in the time series data classification task, and the average classification accuracy is at least 1.1% higher than other types of methods.

[0114] (2) Anomaly detection task

[0115] In this experiment, five mainstream anomaly detection datasets SMD, MSL, SMAP, SWaT, PSM are used, covering practical industrial application fields such as service monitoring, space and earth exploration, and water treatment applications. The experimental results are shown in Table 2, from which it can be seen that the proposed method has achieved the best performance in the anomaly detection task, and has achieved the best results on three of the five experimental datasets. At the same time, the average detection accuracy on the five datasets reached 86.41%, which is higher than other comparison methods.

[0116] Table 2 Abnormality inspection experimental results

[0117]

[0118] (3) Long-term / short-term prediction tasks

[0119] In this verification experiment, in the long-term prediction task, the benchmark data sets used include Electricity, Traffic, Weather, Exchange and ILI, covering the application requirements of four real-world scenarios: electricity, traffic, weather, and finance. Each data set contains a continuous time series data, and the sample data is obtained through a sliding window. In the experiment, the length of the input time series is 96, and the length of the predicted time series is [96,192,336,720]. In the short-term prediction task, the M4 data set contains 100,000 time series data, which is collected through annual, quarterly, monthly, weekly, daily, and time-sharing sampling rates, and covers finance, industry, population and other fields. The length of the predicted sequence in the experiment is [6,8,13,16,24,48].

[0120] The experimental results are shown in Tables 3 and 4, from which it can be seen that the method of the present invention has achieved the best results in both long-term prediction and short-term prediction experiments, which shows that the time-frequency cross-domain unified representation constructed by the present invention has strong semantic representation and generalization capabilities for time series data.

[0121] Table 3 Experimental results of long-term prediction tasks

[0122]

[0123] Table 4 Experimental results of short-term prediction tasks

[0124]

[0125] (4) Missing value completion task

[0126] In this experiment, six groups of experiments were set up, and the data sets used were: ETT, Electricity, Weather. Random occlusion was used to simulate the missing value situation in the experiment, and the random occlusion rate was: [12.5%, 25%, 37.5%, 50%]. The experimental results are shown in Table 5, from which it can be seen that the method of the present invention achieved the best results.

[0127] Table 5 Experimental results of missing value completion task

[0128]

[0129]

[0130] In summary, the present invention provides a time-frequency domain joint unified representation learning method for time series data based on spectrum analysis. The time-frequency domain unified representation model in the present invention has achieved the best results in the five major time series data analysis tasks of classification, anomaly detection, long-term and short-term prediction, and missing value completion. At the same time, the experimental results further show that the time series data representation method proposed in the present invention has extremely strong multi-task adaptability and can excellently complete a variety of complex time series data analysis tasks. Therefore, it can be used as a general network framework for time series data analysis.

[0131] For ease of description, spatially relative terms such as "above", "above", "on the upper surface of", "above", etc. may be used here to describe the spatial positional relationship between a device or feature and other devices or features as shown in the figure. It should be understood that spatially relative terms are intended to include different orientations of the device in use or operation in addition to the orientation described in the figure. For example, if the device in the accompanying drawings is inverted, the device described as "above other devices or structures" or "above other devices or structures" will be positioned as "below other devices or structures" or "below other devices or structures". Thus, the exemplary term "above" can include both "above" and "below". The device can also be positioned in other different ways (rotated 90 degrees or in other orientations), and the spatially relative descriptions used here are interpreted accordingly.

[0132] In addition, it should be noted that the use of terms such as "first" and "second" to limit components is only for the convenience of distinguishing the corresponding components. If not otherwise stated, the above terms have no special meaning and therefore cannot be understood as limiting the scope of protection of the present invention.

[0133] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A time-frequency domain joint unified representation learning method for time series data based on spectrum analysis, characterized in that: The method comprises: Perform feature encoding on the original time series to obtain the encoded time series; The encoded time series is input into the time-frequency joint representation learning model for training, and the unified time-frequency cross-domain representation of the time series data output by the time-frequency joint representation learning model is obtained; wherein the time-frequency joint representation learning model includes L residually connected spectrum joint representation learning modules; during each training, the input data and output data of the current spectrum joint representation learning module are fused, and the fusion result is used as the input data of the next spectrum joint representation learning module; the input data of the first spectrum joint representation learning module is the encoded time series, and the fusion result of the output data and input data of the last spectrum joint representation learning module is the unified time-frequency cross-domain representation of the time series data output by the time-frequency joint representation learning model; The main loss of the time-frequency joint representation learning model is obtained based on the unified time-frequency cross-domain representation of time series data and the labels corresponding to the unified time-frequency cross-domain representation of time series data; Perform 1D convolution operation, linear transformation and normalization operation on the unified time-frequency cross-domain representation of the time series data to obtain the sorting information of the unified time-frequency cross-domain representation of the time series data; The order-preserving loss of the time-frequency joint representation learning model is obtained based on the ordering information of the original time series and the ordering information of the time-frequency cross-domain unified representation of the time series data; The total loss of the time-frequency joint representation learning model is obtained based on the main loss and order-preserving loss of the time-frequency joint representation learning model; Based on the total loss, the structural parameters of the time-frequency joint representation learning model are updated to obtain an updated time-frequency joint representation learning model, and the original time series is continuously trained based on the updated time-frequency joint representation learning model until a preset number of training times is reached to obtain a trained time-frequency joint representation learning model; Collecting the time series corresponding to the application scenario, performing feature encoding on the time series corresponding to the application scenario, and obtaining the original time series corresponding to the application scenario; Based on the trained time-frequency joint representation learning model, the original time series corresponding to the application scenario is uniformly represented across time and frequency domains to obtain a unified time-frequency cross-domain representation of the time series data corresponding to the application scenario.

2. The method according to claim 1, characterized in that The fusion result of the input data and output data of the current spectrum joint representation learning module is obtained by the following formula: In the formula, is the fusion result of the input data and output data of the lth spectral joint representation learning module, is the output data of the lth spectral joint representation learning module, is the input data of the lth spectral joint representation learning module.

3. The method according to claim 1, characterized in that Each spectral joint representation learning module includes a serially connected time-frequency transformation module, a multi-scale parallel complex domain convolutional network, and a feature fusion network; The time-frequency transformation module is used to perform time-frequency transformation on the input data of the current spectrum joint representation learning module to obtain the data after time-frequency transformation; A multi-scale parallel complex domain convolutional network is used to extract features from the data after time-frequency transformation to obtain feature-extracted data; The feature fusion network is used to perform feature fusion on the data after feature extraction to obtain feature fused data, and the feature fused data is the output data of the current spectrum feature extraction and fusion module.

4. The method according to claim 1, characterized in that The time-frequency transformation module is used to perform time-frequency transformation on the input data of the current spectrum joint representation learning module, and the data after time-frequency transformation includes: Performing short-time Fourier transform on the input data of the current spectrum joint representation learning module to obtain several frequency components; A parameter-free complex domain attention mechanism is used to adaptively screen several frequency components to obtain the time-frequency transformed data.

5. The method according to claim 1, characterized in that The multi-scale parallel complex domain convolution network includes several complex domain convolution subnetworks connected in parallel and a splicing module. Each complex domain convolution subnetwork includes two complex domain convolution modules connected in series. The convolution kernel size of each complex domain convolution module is 3×3. The expansion rates of the complex domain convolution modules in different complex domain convolution subnetworks are different, which are 1, 3, 5, 7...; Two complex domain convolution modules in each complex domain convolution subnetwork are used to extract multi-scale features from the time-frequency transformed data to obtain multi-scale features; The splicing module is used to splice multi-scale features to obtain the data after feature extraction.

6. The method according to claim 1, characterized in that The main loss is obtained by the following formula: Loss_main=MSE(Y,Y label ) The order-preserving loss is obtained by the following formula: Loss_series=MSE(X series ,Y series ) The total loss is obtained by the following formula: Loss = Loss_main + Loss_series In the formula, Loss_main is the main loss, MSE(Y,Y label ) is Y, Y label The root mean square error of Y is the unified representation of the time-frequency cross-domain of the time series data, Y label is the time-frequency cross-domain unified representation of the corresponding label of the time series data, Loss_series is the order-preserving loss, MSE(X series , Y series ) is X series , Y series The root mean square error, X series is the sorting information of the original time series, Y series It is the sorting information uniformly represented across time and frequency domains of time series data, and Loss is the total loss.

7. The method according to claim 1, characterized in that The original time series is electricity data, financial data, weather data, medical data, or traffic data.

Images