Training method and device of time series prediction model, equipment and storage medium

Abstract

The invention provides a training method and device of a time series prediction model, equipment and a storage medium, and relates to the technical field of data processing, and the method obtains a time series prediction result based on a multi-head self-attention mechanism after tensor conversion through carrying out feature extraction and graph convolution processing on time series data at different time scales. Time sequence data is subjected to embedded representation, feature extraction is performed at different time scales, and a graph convolution unit is utilized, so that the model can capture inter-sequence correlation related to the scales. And projecting the obtained output into an embedding space with the same dimension as the original dimension, and synchronously capturing the correlation in the time sequence data through a multi-head self-attention mechanism. And finally, carrying out representation learning by utilizing comparative predictive coding. According to the method, correlation between different scales of multivariate time sequence data and correlation inside different scales are captured in an unsupervised mode, and then the prediction accuracy and the generalization ability of the model are improved.

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and specifically to a training method, device, equipment and storage medium for a time series prediction model. Background Art

[0002] Time series prediction has been widely used in the fields of electricity, meteorology, economy, disease, etc. At present, a large number of deep learning models for time series prediction have emerged, such as Transformer, Mamba, RNN, etc.

[0003] Compared with other data types, time series have strong volatility and instability. There are some practical challenges in applying deep learning models to multivariate time series prediction. First, the relationship between multiple variables is multi-scale, and the correlation between series at different time scales may vary significantly due to changes in the market or environment. Existing models fail to effectively capture this complex dynamic correlation between series in different cycles, which constitutes a major problem currently faced. Second, supervised learning is highly dependent on human-labeled data sets for algorithm training in order to more accurately classify data or predict results. This makes supervised learning have high requirements for the quality, accuracy, clustering classification accuracy, etc. of the labeled data sets, and also has a higher threshold for the professional ability and knowledge reserves of personnel involved in the construction of supervised learning models and data labeling. Summary of the invention

[0004] In view of the shortcomings of the related technologies mentioned above, the present application provides a training method, device, equipment and storage medium for a time series prediction model to solve the above technical problems.

[0005] The present application provides a training method for a time series prediction model, wherein the time series prediction model includes an embedding layer, at least two feature extraction modules, and an autoregressive model of contrastive prediction coding, each feature extraction module includes a multi-scale module and a residual connection layer, each multi-scale module includes a feature extraction unit, a graph convolution unit, a multi-head self-attention mechanism, and a multi-layer perceptron, and the method includes: Acquire a training sample set, wherein the training sample set includes a plurality of time series data; Embedding the time series data to obtain embedded feature data; Extracting features from the embedded feature data at different time scales to obtain a plurality of first intermediate data; Performing graph convolution processing on each of the first intermediate data to obtain a plurality of second intermediate data; Performing tensor conversion on each of the second intermediate data based on a multi-layer perceptron to obtain a plurality of third intermediate data; Determine the attention output data corresponding to each third intermediate data based on the multi-head self-attention mechanism; According to all attention output data, multi-scale feature data is obtained; Based on the multi-scale feature data and the autoregressive model of the contrast prediction coding, a time series prediction result is obtained.

[0006] In one embodiment of the present application, the time series data is embedded and represented to obtain embedded feature data, including: Obtaining original sequence embedding, time feature embedding, and position embedding of the time series data; The embedding feature is obtained based on the original sequence embedding, the time feature embedding and the position embedding.

[0007] In one embodiment of the present application, feature extraction is performed on the embedded feature data at different time scales to obtain a plurality of first intermediate data, including: , , , in, represents the embedded feature data, represents the fast Fourier transform of the embedded feature data. It indicates the amplitude value of each frequency point after calculating the fast Fourier transform. Indicates the amplitude at each frequency point Averaging across dimensions, is the dimension of the embedded feature data, vector Contains the average amplitude of all frequencies, represents a real number, represents the size of the lookback window, Indicates time scale, Indicates Amplitude value; Represents the first k maximum values ​​of the amplitude value; Indicates the selection range of amplitude values; Based on the selected time scale set , reshape the input into a three-dimensional tensor and obtain the first intermediate data corresponding to different time scales: , Used to expand a time series by adding zeros in the time dimension, Used to transform data into tensors, Representation based on time scale No. The first intermediate data, Represents the input of the multi-scale module and sets the input of the first multi-scale module .

[0008] In one embodiment of the present application, graph convolution processing is performed on each of the first intermediate data to obtain a plurality of second intermediate data, including: , in, Represents tensor data, , Indicates The weight matrix corresponding to the tensor of time scales, Represents the original feature dimension of time series data; , in, represents the adaptive adjacency matrix, , is the training parameter; Perform graph convolution on tensor data: , in, Indicated in The second intermediate data after fusion of time scales, is the activation function, represents the hyperparameter, Represents the adaptive adjacency matrix Self-driving Second-rate, Indicates that column-level joins are used to join intermediate variables generated during each iteration.

[0009] In one embodiment of the present application, the attention output data corresponding to each third intermediate data is determined based on the multi-head self-attention mechanism, including: , in, represents the attention output data corresponding to the third intermediate data, represents the third intermediate data, Represents the multi-head attention function in the time scale dimension.

[0010] In one embodiment of the present application, multi-scale feature data is obtained based on all attention output data, including: , , in, Represents multi-scale feature data, Represents the average amplitude of each frequency, through Function determines the weight of the third intermediate data .

[0011] To achieve the above-mentioned purpose and other related purposes, the present application provides a training device for a time series prediction model, wherein the time series prediction model includes an embedding layer, at least two feature extraction modules, and an autoregressive model of contrastive prediction coding, each feature extraction module includes a multi-scale module and a residual connection layer, each multi-scale module includes a feature extraction unit, a graph convolution unit, a multi-head self-attention mechanism, and a multi-layer perceptron, and the device includes: A data acquisition module, used to acquire a training sample set, wherein the training sample set includes a plurality of time series data; A first processing module is used to embed the time series data to obtain embedded feature data; A second processing module is used to extract features from the embedded feature data at different time scales to obtain a plurality of first intermediate data; A third processing module, configured to perform graph convolution processing on each of the first intermediate data to obtain a plurality of second intermediate data; a fourth processing module, configured to perform tensor conversion on each of the second intermediate data based on a multi-layer perceptron to obtain a plurality of third intermediate data; A fifth processing module, used to determine the attention output data corresponding to each third intermediate data based on the multi-head self-attention mechanism; A sixth processing module, used for obtaining multi-scale feature data according to all attention output data; The seventh processing module is used to obtain a time series prediction result based on the multi-scale feature data and the autoregressive model of the contrast prediction coding. In one embodiment of the present application, the first processing module includes: An embedding feature acquisition unit, used to acquire the original sequence embedding, time feature embedding and position embedding of the time series data; An embedding feature determination unit is used to obtain the embedding feature based on the original sequence embedding, the time feature embedding and the position embedding.

[0012] To achieve the above-mentioned purpose and other related purposes, the present application also provides an electronic device, which includes one or more processors and one or more memories, wherein at least one computer program is stored in the one or more memories, and the at least one computer program is loaded and executed by the one or more processors to implement the aforementioned training method of one or more time series prediction models.

[0013] To achieve the above-mentioned purpose and other related purposes, the present application also provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor of a computer, the computer executes one or more of the aforementioned training methods for the time series prediction models.

[0014] As described above, the training method, apparatus, device and storage medium of a time series prediction model provided by the present application have the following beneficial effects: A training method for a time series prediction model in the present application, the method extracts features and performs graph convolution processing on time series data at different time scales, and obtains time series prediction results based on a multi-head self-attention mechanism after tensor conversion. By embedding the time series data and extracting features at different time scales, the model is able to capture scale-related inter-sequence correlations using graph convolution units. The obtained output is projected into an embedding space with the same dimension as the original, and the correlation within the time series data can be captured synchronously through a multi-head self-attention mechanism. Finally, representation learning is performed using contrast prediction coding. This method captures the correlations between and within different scales of multivariate time series data in an unsupervised manner, thereby improving the accuracy of predictions and the generalization ability of the model.

[0015] It should be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] The drawings herein are incorporated into the specification and constitute a part of the specification, showing embodiments consistent with the present application, and together with the specification, are used to explain the principles of the present application. Obviously, the drawings described below are only some embodiments of the present application, and for those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative work. In the drawings: Figure 1 is a schematic diagram of the structure of a time series prediction model shown in an exemplary embodiment of the present application; Figure 2 is a flowchart of a training method for a time series prediction model shown in an exemplary embodiment of the present application; Figure 3 It is a structural block diagram of a training device for a time series prediction model shown in an exemplary embodiment of the present application. DETAILED DESCRIPTION

[0017] The following will describe the implementation methods of the present application with reference to the accompanying drawings and preferred embodiments. Those skilled in the art can easily understand other advantages and effects of the present application from the contents disclosed in this specification. The present application can also be implemented or applied through other different specific implementation methods, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present application. It should be understood that the preferred embodiments are only for illustrating the present application, not for limiting the scope of protection of the present application.

[0018] It should be noted that the illustrations provided in the following embodiments are only schematic illustrations of the basic concept of the present application, and thus the drawings only show components related to the present application rather than being drawn according to the number, shape and size of components in actual implementation. In actual implementation, the type, quantity and proportion of each component may be changed at will, and the component layout may also be more complicated.

[0019] In the following description, a large number of details are discussed to provide a more thorough explanation of the embodiments of the present application. However, it is obvious to those skilled in the art that the embodiments of the present application can be implemented without these specific details. In other embodiments, well-known structures and devices are shown in the form of block diagrams rather than in detail to avoid making the embodiments of the present application difficult to understand.

[0020] See also Figure 1 , Figure 1 It is a structural diagram of a time series prediction model shown in an exemplary embodiment of the present application. In the embodiment of the present application, the time series prediction model includes an embedding layer 110, at least two feature extraction modules 120, and an autoregressive model 130 of comparative prediction coding. Each feature extraction module 120 includes a multi-scale module 121 and a residual connection layer 122. Each multi-scale module includes a feature extraction unit, a graph convolution unit, a multi-head self-attention mechanism, and a multi-layer perceptron.

[0021] It should be noted that the input of the residual connection layer includes the input of the multi-scale module 121 and the output of the multi-scale module 122.

[0022] The embedding layer can embed the time series data to obtain embedded feature data. The feature extraction unit can extract features from the embedded feature data at different time scales to obtain multiple first intermediate data. The graph convolution unit can perform graph convolution processing on each first intermediate data to obtain multiple second intermediate data.

[0023] See also Figure 2 , Figure 2 FIG. 1 is a flowchart of a method for training a time series prediction model shown in an exemplary embodiment of the present application. Figure 2It can be seen that the training method of the time series prediction model can include: Step S210, obtaining a training sample set.

[0024] The training sample set includes multiple time series data.

[0025] In one embodiment of the present application, the training sample set may be a training sample set related to the power field, for example, the time series data in the training sample set may be power load data. The training sample set may be a training sample set related to the meteorological field, for example, the time series data in the training sample set may be temperature data. The training sample set may be a training sample set related to the economic field, for example, the time series data in the training sample set may be commodity price data. The training sample set may be a training sample set related to the disease field, for example, the time series data in the training sample set may be symptom monitoring data.

[0026] Step S220, embedding the time series data to obtain embedded feature data.

[0027] In one embodiment of the present application, the original sequence embedding, time feature embedding and position embedding of the time series data can be obtained, and the embedded features are obtained based on the original sequence embedding, time feature embedding and position embedding. Embedding representation can extract features from time series data. The extracted features are more abstract than the original data, which is convenient for reflecting the essential attributes of time series data. Through embedding representation, complex patterns and relationships in time series data can be captured.

[0028] For example, for time series data Embedding representation is performed, including original sequence embedding, time feature embedding and position embedding, and then the three are added together to obtain embedded feature data ,in, represents the size of the lookback window, Indicates the starting position of the prediction window, Represents the original feature dimension of time series data, represents the feature dimension after embedding representation, and the calculation formula is as follows: , in, It is time series data Normalized intermediate sequence data, is a one-dimensional convolution filter (kernel width = 3, step size = 1), with parameters is the balance factor. According to past experience, the default value is 1. is the position embedding, For time embedding, m represents a time frequency, such as minutes, hours, days, weeks, etc., and M represents the total number of time frequencies, which is 5 here, that is, minutes, hours, days, weeks and months are used.

[0029] Step S230: extracting features from the embedded feature data at different time scales to obtain a plurality of first intermediate data.

[0030] In one embodiment of the present application, a Fast Fourier Transform (FFT) may be performed on the embedded feature data to determine a time scale set, which includes a plurality of different time scales. Feature extraction is performed on the embedded feature data based on different time scales to obtain a plurality of first intermediate data. When processing time series data, FFT may reveal frequency components in the data, thereby extracting periodic patterns in the data.

[0031] In one embodiment, step S230 extracts features from the embedded feature data at different time scales to obtain a plurality of first intermediate data, which may include: , , , in, represents the embedded feature data, represents the fast Fourier transform of the embedded feature data. It indicates the amplitude value of each frequency point after calculating the fast Fourier transform. Indicates the amplitude at each frequency point Averaging across dimensions, is the dimension of the embedded feature data, vector Contains the average amplitude of all frequencies, represents a real number, represents the size of the lookback window, Indicates time scale, Indicates Amplitude value; Represents the first k maximum values ​​of the amplitude value; Indicates the selection range of amplitude values.

[0032] Based on the selected time scale set , reshape the input into a three-dimensional tensor and obtain the first intermediate data corresponding to different time scales: , Used to expand a time series by adding zeros in the time dimension, Used to transform data into tensors, Representation based on time scale No. The first intermediate data, Represents the input of the multi-scale module. , set the input of the first multi-scale module , the input of the nth multi-scale module , represents the output of the n-1th multi-scale module, Represents the input of the n-1th multi-scale module.

[0033] Step S240: Perform graph convolution processing on each first intermediate data to obtain multiple second intermediate data.

[0034] In one embodiment of the present application, each first intermediate data may be subjected to graph convolution processing to obtain a plurality of second intermediate data. Graph convolution processing may capture information at different scales of time series data. Through the graph convolution operation, the relationship between different time points may be analyzed at multiple levels, thereby capturing local and global features.

[0035] In one embodiment, the process of performing graph convolution processing on each first intermediate data to obtain a plurality of second intermediate data in step S240 may include: , in, Represents tensor data, , is a learnable weight matrix, representing the The weight matrix corresponding to the tensor of time scales, Represents the original feature dimension of time series data; , in, represents the adaptive adjacency matrix, , are two trainable parameters; The function can be used to normalize the weights between different nodes. h is a custom dimension, which can be 8. Perform graph convolution on tensor data: , in, Indicated in The second intermediate data after fusion of time scales, is the activation function, represents the hyperparameter, Represents the adaptive adjacency matrix Self-driving Second-rate, Indicates that column-level joins are used to join intermediate variables generated during each iteration.

[0036] It should be noted that P is a set of integer adjacency powers, and ‖ represents column-level connection. In the embodiment of the present application, a graph convolution unit with a learnable adjacency matrix is ​​used to enable the model to capture scale-related sequence correlations.

[0037] Step S250, performing tensor conversion on each second intermediate data based on a multi-layer perceptron to obtain a plurality of third intermediate data.

[0038] In one embodiment of the present application, each second intermediate data is converted into a tensor based on a multi-layer perceptron to obtain a plurality of third intermediate data. The second intermediate data can be converted into a tensor to increase the dimension, which can increase the expressive power of the data and enable the subsequent model to better capture the features in the data.

[0039] In one embodiment, a multi-layer perceptron (MLP) may be used to transform the second intermediate data Perform tensor conversion to obtain the third intermediate data , the third intermediate data is a three-dimensional tensor.

[0040] , and Represent the weight and bias at the i-th time scale respectively.

[0041] Step S260, determine the attention output data corresponding to each third intermediate data based on the multi-head self-attention mechanism.

[0042] In one embodiment of the present application, the attention output data corresponding to each third intermediate data can be determined based on the multi-head self-attention mechanism, which can capture the intra-sequence correlation.

[0043] In one embodiment, the process of determining the attention output data corresponding to each third intermediate data based on the multi-head self-attention mechanism in step S260 may include: , in, represents the attention output data corresponding to the third intermediate data, represents the third intermediate data, , Represents the multi-head attention function in the time scale dimension.

[0044] It should be noted that in implementation, This involves changing the size of the input tensor from Reshape A tensor of , B is the batch size.

[0045] Step S270, obtaining multi-scale feature data based on all attention output data.

[0046] In one embodiment of the present application, multi-scale feature data can be obtained based on all attention output data.

[0047] In one embodiment, the process of obtaining multi-scale feature data according to all attention output data in step S270 may include: , , in, Represents multi-scale feature data, Represents the average amplitude of each frequency, through Function determines the weight of the third intermediate data .in, .

[0048] Step S280, obtaining a time series prediction result based on the multi-scale feature data and the autoregressive model of the contrast prediction coding.

[0049] In one embodiment of the present application, contrastive predictive coding (CPC) is a technique for self-supervised learning. An autoregressive model is a statistical modeling technique used to describe the relationship between a future value of a signal and its past value. In an autoregressive model, each observation is composed of a linear combination of the previous observations plus some random error terms. Autoregressive neural networks (such as LSTM and GRU). The autoregressive model of contrastive predictive coding can be an autoregressive model containing a contrastive predictive coding structure. When training the autoregressive model of contrastive predictive coding, the learning parameters of the model can be trained using the contrastive predictive coding method.

[0050] Exemplarily, using contrast prediction coding method on multi-scale feature data In the training learning parameters, multi-scale feature data can be Input into the autoregressive model (such as GRU) of contrastive predictive coding to generate the context representation of the current moment ; Then we can maximize the mutual information I(x;c), which is defined as follows: , Among them, c represents the context representation and x represents the time series data.

[0051] Finally, the time series prediction results are obtained based on the context representation and trained using multi-classification cross entropy loss.

[0052] In summary, the training method of the time series prediction model provided in the embodiment of the present application performs feature extraction and graph convolution processing on the time series data at different time scales, and obtains the time series prediction result based on the multi-head self-attention mechanism after tensor conversion. By embedding the time series data and extracting features at different time scales, the model is able to capture the scale-related inter-sequence correlation using the graph convolution unit. The obtained output is projected into the same embedding space as the original dimension, and the correlation within the time series data can be captured synchronously through the multi-head self-attention mechanism. Finally, representation learning is performed using contrast prediction coding. This method captures the correlation between different scales and within different scales of multivariate time series data in an unsupervised manner, thereby improving the accuracy of prediction and the generalization ability of the model.

[0053] Figure 3 FIG. 1 is a block diagram of a training device for a time series prediction model shown in an exemplary embodiment of the present application. Figure 3 As shown, the exemplary time series prediction model training device 300 includes: The data acquisition module 310 is used to acquire a training sample set, where the training sample set includes a plurality of time series data.

[0054] The first processing module 320 is used to embed the time series data to obtain embedded feature data.

[0055] The second processing module 330 is used to extract features from the embedded feature data at different time scales to obtain a plurality of first intermediate data.

[0056] The third processing module 340 is used to perform graph convolution processing on each first intermediate data to obtain multiple second intermediate data.

[0057] The fourth processing module 350 is used to perform tensor conversion on each second intermediate data based on a multi-layer perceptron to obtain multiple third intermediate data.

[0058] The fifth processing module 360 ​​is used to determine the attention output data corresponding to each third intermediate data based on the multi-head self-attention mechanism.

[0059] The sixth processing module 370 is used to obtain multi-scale feature data according to all attention output data.

[0060] The seventh processing module 380 is used to obtain a time series prediction result based on the multi-scale feature data and the autoregressive model of the contrast prediction coding.

[0061] In one embodiment of the present application, the first processing module includes: An embedding feature acquisition unit, used to acquire original sequence embedding, time feature embedding and position embedding of time series data; The embedding feature determination unit is used to obtain the embedding feature based on the original sequence embedding, the time feature embedding and the position embedding.

[0062] In one embodiment of the present application, the second processing module is further used for: , , , in, represents the embedded feature data, represents the fast Fourier transform of the embedded feature data. It indicates the amplitude value of each frequency point after calculating the fast Fourier transform. Indicates the amplitude at each frequency point Averaging across dimensions, is the dimension of the embedded feature data, vector Contains the average amplitude of all frequencies, represents a real number, represents the size of the lookback window, Indicates time scale, Indicates Amplitude value; Represents the first k maximum values ​​of the amplitude value; Indicates the selection range of amplitude values; Based on the selected time scale set , reshape the input into a three-dimensional tensor and obtain the first intermediate data corresponding to different time scales: , Used to expand a time series by adding zeros in the time dimension, Used to transform data into tensors, Representation based on time scale No. The first intermediate data, Represents the input of the multi-scale module and sets the input of the first multi-scale module .

[0063] In one embodiment of the present application, the third processing module is further used for: , in, Represents tensor data, , Indicates The weight matrix corresponding to the tensor of time scales, Represents the original feature dimension of time series data; , in, represents the adaptive adjacency matrix, , is the training parameter; Perform graph convolution on tensor data: , in, Indicated in The second intermediate data after fusion of time scales, is the activation function, represents the hyperparameter, Represents the adaptive adjacency matrix Self-driving Second-rate, Indicates that column-level joins are used to join intermediate variables generated during each iteration.

[0064] In one embodiment of the present application, the fifth processing module is further configured to: , in, represents the attention output data corresponding to the third intermediate data, represents the third intermediate data, Represents the multi-head attention function in the time scale dimension.

[0065] In one embodiment of the present application, the sixth processing module is further configured to: , , in, Represents multi-scale feature data, Represents the average amplitude of each frequency, through Function determines the weight of the third intermediate data .

[0066] It should be noted that the training device for the time series prediction model provided in the above embodiment and the training method for the time series prediction model provided in the above embodiment belong to the same concept, wherein the specific manner in which each module and unit performs the operation has been described in detail in the method embodiment and will not be repeated here. In practical applications, the training device for the time series prediction model provided in the above embodiment can distribute the above functions to different functional modules as needed, that is, divide the internal structure of the device into different functional modules to complete all or part of the functions described above, and this is not limited here.

[0067] An embodiment of the present application also provides an electronic device, comprising: one or more processors; a storage device for storing one or more programs, when the one or more programs are executed by one or more processors, the electronic device implements the training method of the time series prediction model provided in the above-mentioned embodiments.

[0068] Another aspect of the present application further provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor of a computer, causes the computer to execute the training method of the time series prediction model provided in the above-mentioned embodiments. The computer-readable storage medium may be included in the electronic device described in the above-mentioned embodiments, or may exist independently without being assembled into the electronic device.

[0069] Another aspect of the present application also provides a computer program product or a computer program, which includes a computer instruction stored in a computer-readable storage medium. A processor of a computer device reads the computer instruction from the computer-readable storage medium, and the processor executes the computer instruction, so that the computer device executes the training method of the time series prediction model provided in each of the above embodiments.

[0070] In the embodiments of the present application, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance. "Including" and "comprising" mentioned throughout the specification and claims are open-ended terms and should be interpreted as "including but not limited to".

[0071] The above embodiments are merely illustrative of the principles and effects of the present application, and are not intended to limit the present application. Anyone familiar with the technology may modify or change the above embodiments without violating the spirit and scope of the present application. Therefore, all equivalent modifications or changes made by a person of ordinary skill in the art without departing from the spirit and technical ideas disclosed in the present application shall still be covered by the claims of the present application.

Claims

1. A training method for a time series prediction model, characterized in that: The time series prediction model includes an embedding layer, at least two feature extraction modules, and an autoregressive model of contrastive predictive coding, each feature extraction module includes a multi-scale module and a residual connection layer, each multi-scale module includes a feature extraction unit, a graph convolution unit, a multi-head self-attention mechanism, and a multi-layer perceptron, and the method includes: Acquire a training sample set, wherein the training sample set includes a plurality of time series data; Embedding the time series data to obtain embedded feature data; Extracting features from the embedded feature data at different time scales to obtain a plurality of first intermediate data; Performing graph convolution processing on each of the first intermediate data to obtain a plurality of second intermediate data; Performing tensor conversion on each of the second intermediate data based on a multi-layer perceptron to obtain a plurality of third intermediate data; Determine the attention output data corresponding to each third intermediate data based on the multi-head self-attention mechanism; According to all attention output data, multi-scale feature data is obtained; Based on the multi-scale feature data and the autoregressive model of the contrast prediction coding, a time series prediction result is obtained.

2. The training method of the time series prediction model according to claim 1, characterized in that: The time series data is embedded and represented to obtain embedded feature data, including: Obtaining original sequence embedding, time feature embedding, and position embedding of the time series data; The embedding feature is obtained based on the original sequence embedding, the time feature embedding and the position embedding.

3. The training method of the time series prediction model according to claim 1, characterized in that: Feature extraction is performed on the embedded feature data at different time scales to obtain a plurality of first intermediate data, including: ， ， ， in, represents the embedded feature data, represents the fast Fourier transform of the embedded feature data. It indicates the amplitude value of each frequency point after calculating the fast Fourier transform. Indicates the amplitude at each frequency point Averaging across dimensions, is the dimension of the embedded feature data, vector Contains the average amplitude of all frequencies, represents a real number, represents the size of the lookback window, Indicates time scale, Indicates Amplitude value; Represents the first k maximum values ​​of the amplitude value; Indicates the selection range of amplitude values; Based on the selected time scale set , reshape the input into a three-dimensional tensor and obtain the first intermediate data corresponding to different time scales: ， Used to expand a time series by adding zeros in the time dimension, Used to transform data into tensors, Representation based on time scale No. The first intermediate data, Represents the input of the multi-scale module and sets the input of the first multi-scale module .

4. The training method of the time series prediction model according to claim 1, characterized in that: Performing graph convolution processing on each of the first intermediate data to obtain a plurality of second intermediate data, including: ， in, Represents tensor data, , Indicates The weight matrix corresponding to the tensor of time scales, Represents the original feature dimension of time series data; ， in, represents the adaptive adjacency matrix, , is the training parameter; Perform graph convolution on tensor data: ， in, Indicated in The second intermediate data after fusion of time scales, is the activation function, represents the hyperparameter, Represents the adaptive adjacency matrix Self-driving Second-rate, Indicates that column-level joins are used to join intermediate variables generated during each iteration.

5. The training method of the time series prediction model according to claim 1, characterized in that: The attention output data corresponding to each third intermediate data is determined based on the multi-head self-attention mechanism, including: ， in, represents the attention output data corresponding to the third intermediate data, represents the third intermediate data, Represents the multi-head attention function in the time scale dimension.

6. The training method of the time series prediction model according to claim 1, characterized in that: Based on all attention output data, multi-scale feature data is obtained, including: ， ， in, Represents multi-scale feature data, Represents the average amplitude of each frequency, through Function determines the weight of the third intermediate data .

7. A training device for a time series prediction model, characterized in that: The time series prediction model includes an embedding layer, at least two feature extraction modules, and an autoregressive model of contrastive prediction coding, each feature extraction module includes a multi-scale module and a residual connection layer, each multi-scale module includes a feature extraction unit, a graph convolution unit, a multi-head self-attention mechanism, and a multi-layer perceptron, and the device includes: A data acquisition module, used to acquire a training sample set, wherein the training sample set includes a plurality of time series data; A first processing module is used to embed the time series data to obtain embedded feature data; A second processing module is used to extract features from the embedded feature data at different time scales to obtain a plurality of first intermediate data; A third processing module, configured to perform graph convolution processing on each of the first intermediate data to obtain a plurality of second intermediate data; a fourth processing module, configured to perform tensor conversion on each of the second intermediate data based on a multi-layer perceptron to obtain a plurality of third intermediate data; A fifth processing module, used to determine the attention output data corresponding to each third intermediate data based on the multi-head self-attention mechanism; A sixth processing module, used for obtaining multi-scale feature data according to all attention output data; The seventh processing module is used to obtain a time series prediction result based on the multi-scale feature data and the autoregressive model of the contrast prediction coding.

8. The training device for a time series prediction model according to claim 7, characterized in that: The first processing module includes: An embedding feature acquisition unit, used to acquire the original sequence embedding, time feature embedding and position embedding of the time series data; An embedding feature determination unit is used to obtain the embedding feature based on the original sequence embedding, the time feature embedding and the position embedding.

9. An electronic device, characterized in that: The electronic device includes one or more processors and one or more memories, wherein at least one computer program is stored in the one or more memories, and the at least one computer program is loaded and executed by the one or more processors to implement the training method of the time series prediction model as described in any one of claims 1 to claim 6.

10. A computer-readable storage medium, characterized in that: A computer program is stored thereon, and when the computer program is executed by a processor of a computer, the computer is caused to execute the training method for a time series prediction model as described in any one of claims 1 to 6.

Images