Multivariate time series data processing method and device, equipment and medium

Abstract

The invention relates to the technical field of data processing, and provides a multivariate time series data processing method and device, equipment and a medium, and the method comprises the steps: carrying out the sequence marking of to-be-processed multivariate time series data, and obtaining initial marking data; task marking is carried out on the initial marked data to obtain secondary marked data so as to determine a target processing task corresponding to the multivariate time series data to be processed, and the target processing task comprises at least one of a data prediction task, a data classification task, a missing value interpolation task and an abnormal data monitoring task; and inputting the secondary mark data into a pre-trained multivariate time series data network model, and performing corresponding data processing according to the target processing task to output a task processing result corresponding to the target processing task. Through the technical scheme, fine adjustment or redesign of a task specific module for a new task is not needed, and the generalization ability of the model on the new task is improved.

Description

[Technical field]

[0001] The present application relates to the field of data processing technology, and in particular to a method, device, equipment and medium for processing multivariate time series data. [Background technology]

[0002] The research background of multivariate time series data processing technology is multifaceted. It involves mining potential useful information from multivariate time series and building prediction models, which provides theoretical guidance for control, decision-making and regulation in the fields of nature, medicine, society, industry, etc., and has important practical significance. With the development of data acquisition and storage technology, the dimension and scale of time series data continue to increase, which increases the difficulty of establishing accurate prediction models. Multivariate time series not only focuses on the historical trend of a single variable, but also emphasizes the interaction and influence between variables in order to reveal the inherent laws of the data more comprehensively and accurately. At the same time, multivariate time series data analysis and modeling face many challenges, such as the identification of complex relationships between variables, the processing and storage of large-scale data, and the generalization ability of the model.

[0003] Transformer-based multivariate time series processing technology is one of the current research hotspots. The Transformer model has been introduced into time series analysis, especially the processing of multivariate time series data, due to its success in the field of natural language processing (NLP). Its core advantage is that it can effectively capture long-distance dependencies in sequence data. In time series data analysis, this ability is also important because time series data usually contains complex time-varying features and long-term dependencies.

[0004] iTransformer adopts the encoder architecture of Transformer, including embedding, projection, and Transformer blocks. (a) The original sequences of different variables are embedded as tokens independently. (b) Self-attention is applied to the embedded variable tokens, which enhances interpretability and reveals multivariate correlations. (c) Sequence representations of each token are extracted through a shared feedforward network. (d) Layer normalization is used to reduce the differences between variables.

[0005] The above technical solution has the following technical problems:

[0006] (1) Poor data generalization ability. iTransformer embeds each sequence independently into a variable token to describe multivariate correlation, that is, the data of multiple time points of the same variable is represented as a token. There is no attention mechanism in the time dimension. The data distribution used in training needs to match the data distribution in inference. It cannot adapt to data formats with different rules, which in turn affects the generalization ability of the model.

[0007] (2) It is impossible to achieve self-adaptation of time series tasks. Classification, prediction, filling, anomaly detection and other types of time series tasks have different characteristics and objectives. iTransformer lacks the ability to abstract multi-task features and cannot be completed using a unified model. It is necessary to customize the training model separately to adapt to the task. [Summary of the invention]

[0008] The embodiments of the present application provide a method, device, equipment and medium for processing multivariate time series data, aiming to solve many problems existing in the related art.

[0009] In a first aspect, an embodiment of the present application provides a method for processing multivariate time series data, the method comprising:

[0010] Perform sequence labeling on the multivariate time series data to be processed to obtain initial labeled data;

[0011] Performing task labeling on the initial labeled data to obtain secondary labeled data, so as to determine the target processing task corresponding to the multivariate time series data to be processed, wherein the target processing task includes at least one of the following: a data prediction task, a data classification task, a missing value interpolation task, and an abnormal data monitoring task;

[0012] The secondary labeled data is input into a pre-trained multivariate time series data network model, and corresponding data processing is performed according to the target processing task to output a task processing result corresponding to the target processing task.

[0013] In one embodiment, optionally, performing sequence labeling on the multivariate time series data to be processed to obtain initial labeling data includes:

[0014] Using a non-overlapping patch size p to divide the multivariate time series data to be processed into patches of a preset length L, to obtain a time series z of a length L / p;

[0015] Each patch of the time series z is projected to a preset dimension using a linear layer to perform sequence labeling on the multivariate time series data to be processed to obtain initial labeled data.

[0016] In one embodiment, optionally, performing task marking on the initial marked data to obtain secondary marked data includes:

[0017] According to the target data and specific context required for executing each processing task, task-labeling the initial labeled data to obtain secondary labeled data;

[0018] According to the secondary labeled data, a target processing task corresponding to the multivariate time series data to be processed is determined.

[0019] In one embodiment, optionally, the pre-trained multivariate temporal data network model includes a multi-head self-attention module, a dynamic MLP module, and a gate module connected in sequence;

[0020] Processing the secondary labeled data through the multi-head attention module and outputting processed feature data;

[0021] Performing bilinear interpolation on the data weights corresponding to the processed feature data through the dynamic MLP (Multi-Layer Perceptron) module to generate a corresponding target weight matrix, and performing key feature extraction on the processed feature data according to the target weight matrix to obtain key feature data;

[0022] The key feature data is feature scaled by the gate module to output target key feature data.

[0023] In one embodiment, optionally, performing corresponding data processing according to the target processing task to output a task processing result corresponding to the target processing task includes:

[0024] When the target processing task is a data prediction task, the target key feature data output by the pre-trained multivariate time series data network model is processed according to the pre-constructed mask token to output the corresponding predicted multivariate time series data within a future preset time period;

[0025] When the target processing task is a missing value interpolation task, the missing values ​​of the target key feature data output by the pre-trained multivariate time series data network model are filled according to the pre-constructed mask tokens to output the filled multivariate time series data;

[0026] When the target processing task is an abnormal data monitoring task, the target key feature data output by the pre-trained multivariate time series data network model is filtered according to the pre-built mask token to output abnormal multivariate time series data;

[0027] When the target processing task is a data classification task, category matching is performed on the target key feature data output by the pre-trained multivariate time series data network model according to pre-built classification tokens to output classified multivariate time series data.

[0028] In one embodiment, optionally, the training process of the pre-trained multivariate time series data network model includes:

[0029] Obtain the original multivariate time series data in the training set, perform sequence labeling and task labeling, and obtain the original time series;

[0030] Randomly masking the original time series to generate multiple masked time series;

[0031] Inputting the original time series and the masked time series into the multivariate time series data network model to obtain corresponding point-level representations and sequence-level representations;

[0032] The point-level representations of multiple masked time series are weightedly aggregated based on sequence-level similarity to reconstruct the original time series;

[0033] By calculating the consistency loss, the sequence-level representation distance between the original time series is adjusted to obtain the initial multivariate time series data network model.

[0034] In one embodiment, optionally, the training process of the pre-trained multivariate time series data network model further includes:

[0035] Get a fine-tuning dataset;

[0036] The fine-tuning data set is used to randomly sample data of various processing tasks to perform multi-task joint fine-tuning on the initial multivariate time series data network model to obtain the pre-trained multivariate time series data network model.

[0037] In a second aspect, an embodiment of the present application provides a device for processing multivariate time series data, including:

[0038] A first labeling unit is used to perform sequence labeling on the multivariate time series data to be processed to obtain initial labeling data;

[0039] A second labeling unit is used to perform task labeling on the initial labeled data to obtain secondary labeled data to determine a target processing task corresponding to the multivariate time series data to be processed, wherein the target processing task includes at least one of the following: a data prediction task, a data classification task, a missing value interpolation task, and an abnormal data monitoring task;

[0040] A processing unit is used to input the secondary labeled data into a pre-trained multivariate time series data network model, and perform corresponding data processing according to the target processing task to output a task processing result corresponding to the target processing task.

[0041] In a third aspect, a computer device is provided, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the above-mentioned method for processing multivariate time series data when executing the computer program.

[0042] In a fourth aspect, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the above-mentioned method for processing multivariate time series data are implemented.

[0043] In the scheme implemented by the above-mentioned method, device, equipment and medium for processing multivariate time series data, the multivariate time series data to be processed is sequence labeled to obtain initial labeled data; the initial labeled data is task labeled to obtain secondary labeled data to determine the target processing task corresponding to the multivariate time series data to be processed, wherein the target processing task includes at least one of the following: data prediction task, data classification task, missing value interpolation task and abnormal data monitoring task; the secondary labeled data is input into the pre-trained multivariate time series data network model, and the corresponding data processing is performed according to the target processing task to output the task processing result corresponding to the target processing task. In the present invention, through the time series task token mechanism, and without modifying the model, the task labels output by the model are converted into various types of time series data tasks to achieve unified adaptation of tasks in different fields, such as prediction, classification, interpolation and anomaly detection. There is no need to fine-tune or redesign task-specific modules for new tasks, thereby improving the generalization ability of the model on new tasks. In addition, through the sequence multi-head self-attention, dynamic MLP and gate module processing of the unified network model, the global relationship in the sequence is captured, and time series with different lengths and numbers of variables are processed at the same time; further combined with the unified mask reconstruction pre-training mechanism, cross-domain data generalization is achieved, and various time series processing tasks can be completed with zero-sample and small-sample models.

Brief Description of the Drawings

[0044] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0045] Figure 1 A schematic flowchart of a method for processing multivariate time series data according to an embodiment of the present application is shown.

[0046] Figure 2 A schematic flowchart of the pre-training phase of a pre-trained multivariate temporal data network model according to an embodiment of the present application is shown.

[0047] Figure 3 A schematic flow chart of the fine-tuning phase of a pre-trained multivariate temporal data network model according to an embodiment of the present application is shown.

[0048] Figure 4 A block diagram of a device for processing multivariate time series data according to an embodiment of the present application is shown. [Specific implementation method]

[0049] In order to better understand the technical solution of the present application, the embodiments of the present application are described in detail below with reference to the accompanying drawings.

[0050] It should be clear that the described embodiments are only part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in the field without creative work are within the scope of protection of the present application.

[0051] The terms used in the embodiments of the present application are only for the purpose of describing specific embodiments, and are not intended to limit the present application. The singular forms "a", "said" and "the" used in the embodiments of the present application and the appended claims are also intended to include plural forms, unless the context clearly indicates other meanings.

[0052] In conjunction with the accompanying drawings, some embodiments of the present application are described in detail below. In the absence of conflict, the following embodiments and features in the embodiments can be combined with each other.

[0053] See also Figure 1 , Figure 1 A schematic flow chart of a method for processing multivariate time series data according to an embodiment of the present application is shown. The method for processing multivariate time series data is used to solve many technical problems existing in related technologies.

[0054] like Figure 1 As shown, according to an embodiment of the present application, a method for processing multivariate time series data includes:

[0055] Step S101, sequence labeling is performed on the multivariate time series data to be processed to obtain initial labeling data;

[0056] In one embodiment, optionally, step S101 includes:

[0057] Step S1011, using a non-overlapping patch size p to divide the multivariate time series data to be processed into patches of a preset length L, to obtain a time series z of a length of L / p;

[0058] Step S1012, using a linear layer to project each patch of the time series z to a preset dimension, so as to perform sequence labeling on the multivariate time series data to be processed, and obtain initial labeled data.

[0059] In this embodiment, a tag-based format is used to represent various time series data domain tasks. The tag types include sequence tags, prompt tags, and task tokens. Sequence tags are used to identify input time series samples; prompt tags are used to provide important context about tasks and data to guide the model to complete tasks; task tokens are used to connect prompt tags and sequence tags together, and without modifying the model, convert the task tags output by the model into various time series data tasks, so as to achieve unified adaptation of tasks in different fields.

[0060] First, we construct a sequence token and use a non-overlapping patch size p to divide the input sample into patches of fixed length l, obtaining a time series z of length l / p. We use a linear layer to project each patch of the time series z to a fixed dimension, and then obtain the sequence tag of the input sample. Since different time series often have different variable dimensions, the variable position information in the tag is retained here to adapt to sequence representations of different variable dimensions.

[0061] Step S102, task-marking the initial marked data to obtain secondary marked data, so as to determine the target processing task corresponding to the multivariate time series data to be processed, wherein the target processing task includes at least one of the following: a data prediction task, a data classification task, a missing value interpolation task, and an abnormal data monitoring task;

[0062] In one embodiment, optionally, step S102 includes:

[0063] According to the target data and specific context required for executing each processing task, task-labeling the initial labeled data to obtain secondary labeled data;

[0064] According to the secondary labeled data, a target processing task corresponding to the multivariate time series data to be processed is determined.

[0065] In this step, hint tokens are constructed. Hint learning is defined as learnable embeddings. In a multi-task setting, each task has its own set of hint tokens that contain specific context related to both the data domain and the task that the model needs. The model can be adapted to new tasks through appropriate hint tokens without fine-tuning the model to achieve generalization adaptation to multiple tasks.

[0066] Finally, task tokens are constructed, including mask tokens and classification tokens. Mask tokens are used for generative modeling, covering prediction, interpolation, and anomaly detection. Classification tokens define the general format for representing tasks and support flexible adaptation to new tasks. For prediction tasks, mask tokens are repeated multiple times in the model input for length prediction, and repeated mask tokens in the model output are uniformly processed and converted back to sequences; for classification tasks, the model classification tokens are uniformly processed for embedding matching with categories; for interpolation tasks, missing parts are filled with mask tokens; in anomaly detection, the denoised sequence tokens returned by the model are used to identify abnormal data points.

[0067] Step S103, inputting the secondary labeled data into a pre-trained multivariate time series data network model, and performing corresponding data processing according to the target processing task to output a task processing result corresponding to the target processing task.

[0068] In one embodiment, optionally, the pre-trained multivariate temporal data network model includes a multi-head self-attention module, a dynamic MLP module, and a gate module connected in sequence;

[0069] Processing the secondary labeled data through the multi-head attention module and outputting processed feature data;

[0070] In order to adapt to time series data in different fields and tasks, UniTS uses a multi-head self-attention (MHSA) mechanism in both the sequence and variable dimensions. This allows the model to capture global relationships in the sequence while processing time series of different lengths and numbers of variables. Using the self-attention implementation of the patch sequence, the variable dimension will convolve the Query and Key in the time dimension attention to make an overall self-attention in the variable dimension.

[0071] Performing bilinear interpolation on the data weights corresponding to the processed feature data through the dynamic MLP module to generate a corresponding target weight matrix, and performing key feature extraction on the processed feature data according to the target weight matrix to obtain key feature data;

[0072] In this step, since the lengths of different data inputs are different, the input lengths of the MLP network followed by the traditional attention mechanism are different, and the weight size of the MLP will change. In order to solve this problem, the present invention performs bilinear interpolation on the original MLP weights according to the length of the input vector to generate MLP weights that match the input dimension.

[0073] The key feature data is feature scaled by the gate module to output target key feature data.

[0074] Due to the distribution differences of different data sets, this paper uses a gate module to perform scene-specific scaling on the input features. The scaling factor is generated by passing the input of the previous layer through a sigmoid and multiplied by the output of this layer.

[0075] In one embodiment, optionally, step S103 includes:

[0076] When the target processing task is a data prediction task, the target key feature data output by the pre-trained multivariate time series data network model is processed according to the pre-constructed mask token to output the corresponding predicted multivariate time series data within a future preset time period;

[0077] When the target processing task is a missing value interpolation task, the missing values ​​of the target key feature data output by the pre-trained multivariate time series data network model are filled according to the pre-constructed mask tokens to output the filled multivariate time series data;

[0078] When the target processing task is an abnormal data monitoring task, the target key feature data output by the pre-trained multivariate time series data network model is filtered according to the pre-built mask token to output abnormal multivariate time series data;

[0079] When the target processing task is a data classification task, category matching is performed on the target key feature data output by the pre-trained multivariate time series data network model according to pre-built classification tokens to output classified multivariate time series data.

[0080] Different operations can be performed through task tokens for different target processing tasks. Among them, task tokens include mask tokens and classification tokens. Mask tokens are used for generative modeling, which can cover processing tasks such as prediction, interpolation, and anomaly detection. Classification tokens define the general format for representing tasks and support flexible adaptation to new tasks. For prediction tasks, mask tokens are repeated multiple times in the model input for length prediction, and the repeated mask tokens in the unified processing model output are converted back to sequences; for classification tasks, the unified processing model classification tokens are embedded and matched with categories; for interpolation tasks, the missing parts are filled with mask tokens; in anomaly detection, the denoised sequence tokens returned by the model are used to identify abnormal data points.

[0081] In this embodiment, the iterative training of the multivariate time series data network model includes a pre-training stage and a fine-tuning stage. In the pre-training stage, a reconstruction task is used to partially mask the time series data and then restore it; in the fine-tuning stage, data of various tasks are randomly sampled for multi-task joint fine-tuning, and the classification task uses the cross entropy loss, and the prediction task uses the MSE.

[0082] like Figure 2As shown, in one embodiment, optionally, the pre-training stage of the pre-trained multivariate time series data network model includes:

[0083] Step S201, obtaining the original multivariate time series data in the training set, and performing sequence labeling and task labeling to obtain the original time series;

[0084] Step S202, randomly masking the original time series to generate multiple masked time series;

[0085] Step S203, inputting the original time series and the masked time series into the multivariate time series data network model to obtain corresponding point-level representations and sequence-level representations;

[0086] Step S204, performing weighted aggregation on the point-level representations of multiple masked time series based on the sequence-level similarity to reconstruct the original time series;

[0087] Step S205, by calculating the consistency loss, adjusting the sequence-level representation distance between the original time series to obtain an initial multivariate time series data network model.

[0088] In this embodiment, a unified mask reconstruction pre-training scheme is adopted. First, the original time series is randomly masked to generate multiple masked time series. Then, the original time series and its masked version are input into the model, and their point-level and sequence-level representations are obtained through the encoder and projection layers. The point-level representations of multiple masked time series are weightedly aggregated based on the sequence-level similarity to reconstruct the original time series. Finally, the consistency loss is introduced, and the sequence-level representation distance between positive and negative samples is adjusted through the contrastive learning strategy, and the aggregation weights between multiple masked sequences are controlled based on the similarity between the original time series and the masked sequence, so as to realize a learnable neighbor weight aggregation process.

[0089] like Figure 3 As shown, in one embodiment, optionally, the fine-tuning phase of the pre-trained multivariate time series data network model includes:

[0090] Step S301, obtaining a fine-tuning data set;

[0091] Step S302, using the fine-tuning data set, randomly sampling data of various processing tasks to perform multi-task joint fine-tuning on the initial multivariate time series data network model to obtain the pre-trained multivariate time series data network model.

[0092] In this embodiment, after pre-training is completed, the model is fine-tuned on a specific downstream task. This involves further training the model on the target dataset to adapt to specific task requirements, reduce resource requirements caused by parameter updates, and avoid catastrophic forgetting; depending on the type of fine-tuning task, such as prediction, classification, anomaly detection, etc., a task-specific head network is added and trained to test the model's ability to handle sequences of different lengths and its generalization in multi-step rolling prediction scenarios.

[0093] After training the pre-trained multivariate time series data network model, it can be further evaluated online. The multivariate time series data network model is mainly developed from the dimensions of model performance, accuracy and generalization ability. First, the mean square error (MSE), mean absolute error (MAE), mean absolute percentage error (MAPE), and root mean square error (RMSE) are selected as evaluation indicators; secondly, the data set is divided into training set, validation set and test set, and k-fold cross-validation and time series split (TimeSeriesSpl it) are selected as validation methods; finally, a rolling forecast evaluation is performed, the forecast window is continuously updated, the model is retrained and forecasted to evaluate the performance of the model over time, and the stability of the model at different time points is evaluated by comparing whether the distribution ratio of the model scores of the training sample and the test sample is consistent.

[0094] The above technical solution of the invention is compared and analyzed with iTransformer, TimesNet and PatchTST, mainly focusing on seven data sets, including four ETT data sets (ETTh1, ETTh2, ETTm1, ETTm2), weather, traffic and electricity (Wu et al., 2021). Four types of tasks, namely prediction, filling and anomaly detection, are verified on each data set, for a total of 28 tasks. In each type of task test, the technical solution of the present invention achieved the best average score. In the prediction task, the technical solution of the present invention leads the second-best iTransformer by 5.2% in MSE index, and the technical solution of the present invention leads the second-best PatchTST by 3.8% in MAE index; in the filling task, the technical solution of the present invention leads the second-best iTransformer by 4.8% in MSE index, and the technical solution of the present invention leads the second-best iTransformer by 2.5% in MAE index; in the anomaly detection task, the technical solution of the present invention leads the second-best TimesNet by 2.8% in MSE index, and the technical solution of the present invention leads the second-best PatchTST by 3.5% in MAE index. In summary, the technical solution of the present invention has a strong application effect in terms of unified time series field data and task diversity.

[0095] It should be understood that the order of execution of the steps in the above embodiment does not necessarily mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiment of the present invention.

[0096] Figure 4 A block diagram of a device for processing multivariate time series data according to an embodiment of the present application is shown.

[0097] like Figure 4 As shown, in a second aspect, an embodiment of the present application provides a multivariate time series data processing device 40, comprising:

[0098] A first labeling unit 41 is used to perform sequence labeling on the multivariate time series data to be processed to obtain initial labeling data;

[0099] A second labeling unit 42 is used to perform task labeling on the initial labeled data to obtain secondary labeled data to determine a target processing task corresponding to the multivariate time series data to be processed, wherein the target processing task includes at least one of the following: a data prediction task, a data classification task, a missing value interpolation task, and an abnormal data monitoring task;

[0100] The processing unit 43 is used to input the secondary labeled data into the pre-trained multivariate time series data network model, and perform corresponding data processing according to the target processing task to output the task processing result corresponding to the target processing task.

[0101] In one embodiment, optionally, the first marking unit is used to:

[0102] Using a non-overlapping patch size p to divide the multivariate time series data to be processed into patches of a preset length L, to obtain a time series z of a length L / p;

[0103] Each patch of the time series z is projected to a preset dimension using a linear layer to perform sequence labeling on the multivariate time series data to be processed to obtain initial labeled data.

[0104] In one embodiment, optionally, the second marking unit is used to:

[0105] According to the target data and specific context required for executing each processing task, task-labeling the initial labeled data to obtain secondary labeled data;

[0106] According to the secondary labeled data, a target processing task corresponding to the multivariate time series data to be processed is determined.

[0107] In one embodiment, optionally, the pre-trained multivariate temporal data network model includes a multi-head self-attention module, a dynamic MLP module, and a gate module connected in sequence;

[0108] Processing the secondary labeled data through the multi-head attention module and outputting processed feature data;

[0109] Performing bilinear interpolation on the data weights corresponding to the processed feature data through the dynamic MLP module to generate a corresponding target weight matrix, and performing key feature extraction on the processed feature data according to the target weight matrix to obtain key feature data;

[0110] The key feature data is feature scaled by the gate module to output target key feature data.

[0111] In one embodiment, optionally, the processing unit is configured to:

[0112] When the target processing task is a data prediction task, the target key feature data output by the pre-trained multivariate time series data network model is processed according to the pre-constructed mask token to output the corresponding predicted multivariate time series data within a future preset time period;

[0113] When the target processing task is a missing value interpolation task, the missing values ​​of the target key feature data output by the pre-trained multivariate time series data network model are filled according to the pre-constructed mask tokens to output the filled multivariate time series data;

[0114] When the target processing task is an abnormal data monitoring task, the target key feature data output by the pre-trained multivariate time series data network model is filtered according to the pre-built mask token to output abnormal multivariate time series data;

[0115] When the target processing task is a data classification task, category matching is performed on the target key feature data output by the pre-trained multivariate time series data network model according to pre-built classification tokens to output classified multivariate time series data.

[0116] In one embodiment, optionally, the training process of the pre-trained multivariate time series data network model includes:

[0117] Obtain the original multivariate time series data in the training set, perform sequence labeling and task labeling, and obtain the original time series;

[0118] Randomly masking the original time series to generate multiple masked time series;

[0119] Inputting the original time series and the masked time series into the multivariate time series data network model to obtain corresponding point-level representations and sequence-level representations;

[0120] The point-level representations of multiple masked time series are weightedly aggregated based on sequence-level similarity to reconstruct the original time series;

[0121] By calculating the consistency loss, the sequence-level representation distance between the original time series is adjusted to obtain the initial multivariate time series data network model.

[0122] In one embodiment, optionally, the training process of the pre-trained multivariate time series data network model further includes:

[0123] Get a fine-tuning dataset;

[0124] The fine-tuning data set is used to randomly sample data of various processing tasks to perform multi-task joint fine-tuning on the initial multivariate time series data network model to obtain the pre-trained multivariate time series data network model.

[0125] For the specific definition of the processing device for multivariate time series data, please refer to the definition of the processing method for multivariate time series data in the above text, which will not be repeated here. Each module in the above-mentioned processing device for multivariate time series data can be implemented in whole or in part by software, hardware and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0126] In one embodiment, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the method described in the first aspect of the embodiment is implemented when the processor executes the computer program.

[0127] In one embodiment, a computer-readable storage medium is provided, which stores a computer program. When the computer program is executed by a processor, the steps of the above-mentioned method for processing multivariate time series data are implemented.

[0128] It should be noted that the above functions or steps that can be implemented by the computer-readable storage medium or electronic device can refer to the relevant description in the aforementioned method embodiment. To avoid repetition, they will not be described one by one here.

[0129] It should be understood that the term "and / or" used in this article is only a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article generally indicates that the associated objects before and after are in an "or" relationship.

[0130] It should be understood that, although the terms first, second, etc. may be used to describe the setting unit in the embodiments of the present application, these setting units should not be limited to these terms. These terms are only used to distinguish the setting units from each other. For example, without departing from the scope of the embodiments of the present application, the first setting unit may also be referred to as the second setting unit, and similarly, the second setting unit may also be referred to as the first setting unit.

[0131] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0132] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of hardware plus software functional units.

[0133] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0134] The embodiments described above are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included in the protection scope of the present invention.

Claims

1. A method for processing multivariate time series data, characterized in that: The method comprises: Perform sequence labeling on the multivariate time series data to be processed to obtain initial labeled data; Performing task labeling on the initial labeled data to obtain secondary labeled data, so as to determine the target processing task corresponding to the multivariate time series data to be processed, wherein the target processing task includes at least one of the following: a data prediction task, a data classification task, a missing value interpolation task, and an abnormal data monitoring task; The secondary labeled data is input into a pre-trained multivariate time series data network model, and corresponding data processing is performed according to the target processing task to output a task processing result corresponding to the target processing task.

2. The method according to claim 1, characterized in that The step of performing sequence labeling on the multivariate time series data to be processed to obtain initial labeling data includes: Using a non-overlapping patch size p to divide the multivariate time series data to be processed into patches of a preset length L, to obtain a time series z of a length L / p; Each patch of the time series z is projected to a preset dimension using a linear layer to perform sequence labeling on the multivariate time series data to be processed to obtain initial labeled data.

3. The method according to claim 1, characterized in that The step of performing task marking on the initial marked data to obtain secondary marked data includes: According to the target data and specific context required for executing each processing task, task-labeling the initial labeled data to obtain secondary labeled data; According to the secondary labeled data, a target processing task corresponding to the multivariate time series data to be processed is determined.

4. The method according to claim 1, characterized in that: The pre-trained multivariate temporal data network model includes a multi-head self-attention module, a dynamic MLP module, and a gate module connected in sequence; Processing the secondary labeled data through the multi-head attention module and outputting processed feature data; Performing bilinear interpolation on the data weights corresponding to the processed feature data through the dynamic MLP module to generate a corresponding target weight matrix, and performing key feature extraction on the processed feature data according to the target weight matrix to obtain key feature data; The key feature data is feature scaled by the gate module to output target key feature data.

5. The method according to claim 4, characterized in that Performing corresponding data processing according to the target processing task to output a task processing result corresponding to the target processing task includes: When the target processing task is a data prediction task, the target key feature data output by the pre-trained multivariate time series data network model is processed according to the pre-constructed mask token to output the corresponding predicted multivariate time series data within a future preset time period; When the target processing task is a missing value interpolation task, the missing values ​​of the target key feature data output by the pre-trained multivariate time series data network model are filled according to the pre-constructed mask tokens to output the filled multivariate time series data; When the target processing task is an abnormal data monitoring task, the target key feature data output by the pre-trained multivariate time series data network model is filtered according to the pre-built mask token to output abnormal multivariate time series data; When the target processing task is a data classification task, category matching is performed on the target key feature data output by the pre-trained multivariate time series data network model according to pre-built classification tokens to output classified multivariate time series data.

6. The method according to claim 1, characterized in that The training process of the pre-trained multivariate time series data network model includes: Obtain the original multivariate time series data in the training set, perform sequence labeling and task labeling, and obtain the original time series; Randomly masking the original time series to generate multiple masked time series; Inputting the original time series and the masked time series into the multivariate time series data network model to obtain corresponding point-level representations and sequence-level representations; The point-level representations of multiple masked time series are weightedly aggregated based on sequence-level similarity to reconstruct the original time series; By calculating the consistency loss, the sequence-level representation distance between the original time series is adjusted to obtain the initial multivariate time series data network model.

7. The method according to claim 6, characterized in that The training process of the pre-trained multivariate time series data network model also includes: Get a fine-tuning dataset; The fine-tuning data set is utilized to randomly sample data of various processing tasks to perform multi-task joint fine-tuning on the initial multivariate time series data network model to obtain the pre-trained multivariate time series data network model.

8. A device for processing multivariate time series data, characterized in that: include: A first labeling unit is used to perform sequence labeling on the multivariate time series data to be processed to obtain initial labeling data; A second labeling unit is used to perform task labeling on the initial labeled data to obtain secondary labeled data to determine a target processing task corresponding to the multivariate time series data to be processed, wherein the target processing task includes at least one of the following: a data prediction task, a data classification task, a missing value interpolation task, and an abnormal data monitoring task; A processing unit is used to input the secondary labeled data into a pre-trained multivariate time series data network model, and perform corresponding data processing according to the target processing task to output a task processing result corresponding to the target processing task.

9. A computer device, characterized in that: include: at least one processor; and, a memory communicatively coupled to the at least one processor; The memory stores instructions executable by the at least one processor, wherein the instructions are configured to execute the method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: Computer executable instructions are stored, and the computer executable instructions are used to execute the method according to any one of claims 1 to 7.

Images