Multi-scale correlation time sequence prediction method, system, device, product and medium
Through the multi-scale correlation time series prediction method, the time encoder and variable encoder are used to extract features, and the feature fusion is performed through the stage conversion layer and the multi-head attention mechanism, which solves the problem that the existing methods fail to fully utilize the timing mode and variable-related features, and realizes high-precision and multi-dimensional timing prediction.
Patent Information
- Application Number
- CN202411892339.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-12-20
- Publication Date
- 2025-03-18
AI Technical Summary
The existing time series prediction methods fail to make full use of timing mode and variable-related feature information, and fail to effectively combine information from multiple dimensions and consider the differences in timing information in temporal order and data tendency characteristics.
Through the multi-scale correlation time series prediction method, the input sequence is obtained and split and stationary. The time encoder and variable encoder are used to extract time characteristics and data tendency characteristics respectively, and multiple feature extraction and fusion are performed through the stage conversion layer and the multi-head attention mechanism, and finally obtain high-precision timing prediction results.
It improves the accuracy of timing prediction and the comprehensiveness of data dimensions, effectively reduces noise in the data, and realizes multi-dimensional timing prediction.
Smart Images
Figure CN119646451A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a multi-scale associated time series prediction method, system, equipment, product and medium. Background Art
[0002] Time series prediction, also known as time series forecasting, is one of the most important tasks of time series analysis. Time series forecasting is to predict the value or pattern of data at a future moment or time period based on the existing data observations sorted in chronological order. Time series forecasting has important research and application value and is widely used in various fields such as weather, economy, energy, and transportation.
[0003] Since the time series prediction methods based on traditional methods have problems such as insufficient learning ability, the vigorous development of deep learning models has opened up new research routes for time series prediction tasks, and Transformer is a typical representative. Transformer is a new neural network architecture for processing sequence data. Transformer has excellent performance in time series prediction tasks. However, Transformer still has two problems: First, the existing methods fail to fully utilize the time series patterns and variable-related feature information of time series. Specifically, most of the work based on Transformer only models a single information dimension and ignores other information dimensions. Although a few works recognize the importance of combining multiple information dimensions, they still cannot effectively combine information from multiple dimensions. Second, the existing methods do not take into account the differences in time series information in terms of time order and data tendency characteristics. Specifically, although a few works consider the association between time order and data tendency characteristics at the same time, they often use a single standard to cross-extract time order and data tendency characteristics. Summary of the invention
[0004] The present invention aims to solve at least one of the technical problems existing in the related art. To this end, the present invention provides a multi-scale correlation time series prediction method to achieve time series prediction with high prediction accuracy.
[0005] The present invention provides a multi-scale correlation time series prediction method, comprising: S1: Obtain an input sequence, split the input sequence to obtain multiple single-dimensional input sequences, and calculate the stationary time series and single-dimensional non-stationary data of each single-dimensional input sequence; S2: Obtain a position coding enhancement vector, cut each of the smoothed time series to obtain multiple slice time series, embed each of the slice time series to obtain multiple initial time series vectors, and add each of the initial time series vectors to the position coding enhancement vector to obtain multiple time series vectors; S3: construct a time encoder of a multi-head attention mechanism, input all the time series vectors into the time encoder in sequence, and calculate multiple time code sequences; S4: constructing a phase conversion layer, inputting all the time coding sequences into the phase conversion layer, and performing multiple tendency feature extractions on all the time coding sequences through the phase conversion layer to obtain multiple feature time series; S5: construct a variable encoder of a multi-head attention mechanism, input all the feature time series into the variable encoder, and calculate multiple variable encoding sequences; S6: Reorganize and expand all the variable coding sequences to obtain multiple single-dimensional prediction results, obtain multiple target single-dimensional prediction results through multiple single-dimensional non-stationary data and multiple single-dimensional prediction results, and fuse all the target single-dimensional prediction results to obtain the target prediction result.
[0006] According to the multi-scale correlation time series prediction method provided by the present invention, step S1 specifically includes: S11: Acquire an input sequence, determine the data dimension of the data in the input sequence, and split the input sequence according to the data dimension to obtain a plurality of single-dimensional input sequences; S12: Calculate the dimensional data mean and dimensional data standard deviation of each of the single-dimensional input sequences, calculate the stationary time series of each of the single-dimensional input sequences through the dimensional data mean and the dimensional data standard deviation, and use the dimensional data mean and the dimensional data standard deviation as the single-dimensional non-stationary data of the single-dimensional input sequence.
[0007] According to the multi-scale correlation time series prediction method provided by the present invention, step S3 specifically includes: S31: Establish a time encoder of a multi-head attention mechanism, wherein the time encoder includes a plurality of identical time encoder layers connected in sequence; S32: input all the time series vectors into the first time encoder layer in sequence; S33: in the first time encoder layer, sequentially calculating a time query vector, a time key vector and a time value vector of each time series vector, and outputting the time query vector, the time key vector and the time value vector as an initial time encoder layer; S34: performing time layer output processing on all the initial time encoder layer outputs to obtain a plurality of the time encoder layer outputs, and inputting all the time encoder layer outputs into the next time encoder layer; S35: Output the time encoder layer of the last time encoder layer as the time encoding sequence.
[0008] According to the multi-scale correlation time series prediction method provided by the present invention, step S4 specifically includes: S41: constructing the phase conversion layer including a plurality of feature extraction matrices, and inputting all the time coding sequences into the phase conversion layer; S42: In the stage conversion layer, multiple fusion and tendency feature extraction are performed on all the time coding sequences through multiple feature extraction matrices to obtain multiple feature time series.
[0009] According to the multi-scale correlation time series prediction method provided by the present invention, step S5 specifically includes: S51: Establish a variable encoder of a multi-head attention mechanism, wherein the variable encoder includes a plurality of identical variable encoder layers connected in sequence; S52: inputting all the feature time series into the first variable encoder layer in sequence; S53: in the first variable encoder layer, sequentially calculating the variable query vector, the variable key vector and the variable value vector of each of the feature time series, and outputting the variable query vector, the variable key vector and the variable value vector as the initial variable encoder layer; S54: performing variable layer output processing on all the initial variable encoder layer outputs to obtain a plurality of the variable encoder layer outputs; and inputting all the variable encoder layer outputs into the next variable encoder layer; S55: Use the variable encoder layer output of the last variable encoder layer as the variable encoding sequence.
[0010] According to the multi-scale correlated time series prediction method provided by the present invention, in step S2, before cutting the smoothed time series, first determine the slicing interval S, and then repeat the last value of the smoothed time series S times to fill the tail of each smoothed time series.
[0011] The present invention also provides a multi-scale correlation time series prediction system, comprising: Data stabilization module: used to obtain an input sequence, split the input sequence to obtain multiple single-dimensional input sequences, and calculate the stabilized time series and single-dimensional non-stationary data of each single-dimensional input sequence; Time series vector module: used to obtain the position coding enhancement vector, cut each of the smoothed time series to obtain multiple sliced time series, embed each of the sliced time series to obtain multiple initial time series vectors, and add each of the initial time series vectors to the position coding enhancement vector to obtain multiple time series vectors; Time encoder module: used to construct a time encoder of a multi-head attention mechanism, input all the time series vectors into the time encoder in sequence, and calculate multiple time code sequences; Phase conversion layer module: used to construct a phase conversion layer, input all the time code sequences into the phase conversion layer, and perform multiple tendency feature extractions on all the time code sequences through the phase conversion layer to obtain multiple feature time series; Variable encoder module: used to construct a variable encoder of a multi-head attention mechanism, input all the feature time series into the variable encoder, and calculate multiple variable encoding sequences; Target prediction result module: used to reorganize and expand the projection processing of all the variable coding sequences to obtain multiple single-dimensional prediction results, obtain multiple target single-dimensional prediction results through multiple single-dimensional non-stationary data and multiple single-dimensional prediction results, and fuse all the target single-dimensional prediction results to obtain the target prediction result.
[0012] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the steps of any of the multi-scale correlation time series prediction methods described above are implemented.
[0013] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of any of the multi-scale correlation time series prediction methods described above.
[0014] The present invention also provides a computer program product, which includes a computer program stored on a non-transitory computer-readable storage medium, and the computer program includes program instructions. When the program instructions are executed by a computer, the computer can execute the steps of any of the multi-scale correlation time series prediction methods described above.
[0015] The above one or more technical solutions in the embodiments of the present invention have at least one of the following technical effects: The multi-scale correlation time series prediction method, system, device, product and medium provided by the present invention first stabilize the input sequence, and first extract the time feature by using the time encoder, and then extract the data tendency feature by using the variable encoder, rather than cross extraction, effectively reducing the noise in the data. Then the present invention performs multi-dimensional time series prediction by combining the data tendency feature and the time feature, effectively improving the accuracy of the time series prediction, and the data dimension of the time series prediction is also higher, so the data content is more comprehensive.
[0016] Additional aspects and advantages of the present invention will be given in part in the following description and in part will be obvious from the following description, or will be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS
[0017] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.
[0018] Figure 1 It is a flow chart of the multi-scale correlation time series prediction method provided by the present invention.
[0019] Figure 2 It is a structural schematic diagram of the multi-scale correlation time series prediction system provided by the present invention.
[0020] Figure 3 It is a structural schematic diagram of the multi-scale correlation time series prediction device provided by the present invention.
[0021] Reference numerals: 100, data stabilization module; 200, time series vector module; 300, time encoder module; 400, stage conversion layer module; 500, variable encoder module; 600, target prediction result module; 810, processor; 820, communication interface; 830, memory; 840, communication bus. DETAILED DESCRIPTION
[0022] In order to make the purpose, technical scheme and advantages of the present invention clearer, the technical scheme in the present invention will be clearly and completely described below. Obviously, the described embodiments are part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments in the present invention, all other embodiments obtained by ordinary technicians in the field without creative work are within the scope of protection of the present invention. The following embodiments are used to illustrate the present invention, but cannot be used to limit the scope of the present invention.
[0023] In the description of the embodiments of the present invention, it should be noted that the terms “first”, “second” and “third” are only used for descriptive purposes and cannot be understood as indicating or implying relative importance.
[0024] In the description of the embodiments of the present invention, it should be noted that, unless otherwise clearly specified and limited, the terms "connected" and "connection" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection or an indirect connection through an intermediate medium. For ordinary technicians in this field, the specific meanings of the above terms in the embodiments of the present invention can be understood according to specific circumstances.
[0025] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the embodiment of the present invention. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, without contradiction.
[0026] Combine the following Figures 1 to 3 Describing embodiments of the present invention: Figure 1 This is a flow chart of the multi-scale associated time series prediction method. First, the input sequence is obtained, and then the stationary time series is obtained according to the input sequence. Subsequently, the single-dimensional prediction result can be obtained through the time encoder and the variable encoder. That is, the stationary time series is first cut and embedded to obtain the time series vector, and the time series vector is input into the time encoder, and then passed through the stage conversion layer and the variable encoder, and the target single-dimensional prediction result is obtained after the reorganization, expansion and projection processing, and finally the target prediction result is obtained.
[0027] The present invention provides a multi-scale correlation time series prediction method, comprising: S1: Obtain an input sequence, split the input sequence to obtain multiple single-dimensional input sequences, and calculate the stationary time series and single-dimensional non-stationary data of each single-dimensional input sequence; Furthermore, the purpose of this stage is to split the input sequence into single-dimensional input sequences and perform stationary processing on each single-dimensional input sequence, thereby obtaining a stationary time series and single-dimensional non-stationary data. Step S1 further includes: S11: Acquire an input sequence, determine the data dimension of the data in the input sequence, and split the input sequence according to the data dimension to obtain a plurality of single-dimensional input sequences; S12: Calculate the dimensional data mean and dimensional data standard deviation of each of the single-dimensional input sequences, calculate the stationary time series of each of the single-dimensional input sequences through the dimensional data mean and the dimensional data standard deviation, and use the dimensional data mean and the dimensional data standard deviation as the single-dimensional non-stationary data of the single-dimensional input sequence.
[0028] With respect to the above steps, the specific implementation methods in this embodiment are as follows: First, historical data including time data is obtained as an input sequence, and then the data dimension of the data in the input sequence is determined. The data dimension is the type of data contained in the input sequence other than time data. For example, when the input sequence we obtain is weather data over a period of time, and the weather data includes maximum temperature, minimum temperature, humidity and cloud cover, the maximum temperature, minimum temperature, humidity and cloud cover can be considered as the four data dimensions of the input sequence. Then, the input sequence is split according to the data dimension to obtain multiple single-dimensional input sequences. For example, after splitting the weather data described above, four single-dimensional input sequences including maximum temperature, minimum temperature, humidity and cloud cover can be obtained.
[0029] In getting the One-dimensional input sequence ,in And D is the number of data dimensions, the mean of its dimensional data can be calculated And the standard deviation of dimension data : Among them, L is the lookback window length, that is, the time length of the input sequence, which can be represented by the number of data contained in the input sequence. For example, when the weather data mentioned above is measured once a day and the time length is 30 days, each single-dimensional input sequence contains 30 weather data, and the lookback window length is 30. Indicates The first of the one-dimensional input sequences data.
[0030] After obtaining the dimension data mean and dimension data standard deviation of each single-dimensional input sequence, the dimension data mean and dimension data standard deviation can be used as the single-dimensional non-stationary data of the single-dimensional input sequence. The first one in the stationary time series of the single-dimensional input series Stabilized time data : in, is the first learnable stabilization parameter, is a learnable second stabilization parameter. In this embodiment, learnable means that the parameter has an initial value, and the parameter can be considered as a constant each time it participates in the operation. After the operation is completed, the parameter will optimize and iterate itself, so that its value is more reasonable and the operation accuracy can be effectively improved. is a stable constant. This parameter is a fixed value, which can avoid the stationary time series being too large due to the denominator of the division formula being too small. After all the stationary time data in the stationary time series are calculated, the calculation of the stationary time series can be completed.
[0031] S2: Obtain a position coding enhancement vector, cut each of the smoothed time series to obtain multiple slice time series, embed each of the slice time series to obtain multiple initial time series vectors, and add each of the initial time series vectors to the position coding enhancement vector to obtain multiple time series vectors; Furthermore, the purpose of this stage is to cut and embed each of the smoothed time series to obtain multiple initial time series vectors, and then add each of the initial time series vectors to the position coding enhancement vector to obtain multiple time series vectors.
[0032] Specifically, first, before cutting each of the stationary time series, it is necessary to determine the slice interval S, that is, how many data in the stationary time series are required for each cut. Then, the last value of the stationary time series is repeated S times to fill the tail of the stationary time series, so as to avoid the loss of the tail information of the stationary time series during the cutting process. At this time, the number of slices can be calculated And cut according to the number of slices to get multiple slice time series: Here, P is the slice length, that is, the length of each slice. It should be noted that the slice length and the slice interval can be different. For example, there is a set of data 1, 2, 3, 4, 5 and they are cut. When the slice length is set to 2 and the slice interval is 1, the results of the cutting are: 1, 2; 2, 3; 3, 4; 4, 5. In addition, the slice length should not be too large or too small. If the slice length is less than the slice interval, there is a part of the stable time data that is not included in any slice time series, which leads to a decrease in prediction accuracy due to information loss; when the slice length is much larger than the slice interval, the adjacent slice time series have too large similarities due to the aggregation of a large amount of repeated information, which also leads to a decrease in prediction accuracy.
[0033] Finally, each slice time series is embedded, that is, each slice time series is respectively associated with a learnable linear mapping matrix By multiplying them, we can get multiple initial time series vectors. Since the initial time series vector lacks information about the position, it is necessary to obtain a position coding enhancement vector and add each initial time series vector to the position coding enhancement vector to get multiple time series vectors.
[0034] S3: construct a time encoder of a multi-head attention mechanism, input all the time series vectors into the time encoder in sequence, and calculate multiple time code sequences; Furthermore, the purpose of this stage is to calculate the temporal feature relationship between the time series vectors through the time encoder and capture the temporal feature information between the time series vectors. Specifically, step S3 further includes: S31: Establish a time encoder of a multi-head attention mechanism, wherein the time encoder includes a plurality of identical time encoder layers connected in sequence; S32: input all the time series vectors into the first time encoder layer in sequence; S33: in the first time encoder layer, sequentially calculating a time query vector, a time key vector and a time value vector of each time series vector, and outputting the time query vector, the time key vector and the time value vector as an initial time encoder layer; S34: performing time layer output processing on all the initial time encoder layer outputs to obtain a plurality of the time encoder layer outputs, and inputting all the time encoder layer outputs into the next time encoder layer; S35: Output the time encoder layer of the last time encoder layer as the time encoding sequence.
[0035] With respect to the above steps, the specific implementation methods in this embodiment are as follows: First, based on the encoder structure in Transformer, a time encoder with a multi-head attention mechanism is constructed, and the time encoder includes multiple identical time encoder layers connected in sequence. Then all time series vectors are input into the first time encoder layer in sequence. In the first time encoder layer, the first The first dimension time series vector In the multi-head attention mechanism Time query vector of each head , time key vector and the time value vector : in, For the The first dimension A time series vector, For the learnable The linear mapping matrix of the time query vector of each head, For the learnable The linear mapping matrix of the time key vector of each head, The first The linear mapping matrix of the time value vector of each head, here, , and It can be used by data of different dimensions, thus saving the space occupied by the model and implicitly learning the connection between data of different dimensions. The time query vector, time key vector and time value vector of each time series vector in each head are calculated in turn and output as the initial time encoder layer.
[0036] Then, all the initial time encoder layer outputs are processed for the time layer output, that is, firstly, the time query vector, time key vector and time value vector of each head are calculated to obtain the first time encoder in the time encoder. The first dimension The time series vector The temporal attention mechanism output of each head : in, () is for softmax transformation, T represents transposition, , represents the temporal attention mechanism coefficient, is the number of heads in the multi-head attention mechanism in the temporal encoder layer, for After obtaining the temporal attention mechanism output of each head, the temporal attention mechanism output of each head is fused to obtain the first The first dimension The temporal attention mechanism outputs a time series vector , thus obtaining The first dimension The initial temporal encoder layer outputs a time series vector : in, The mapping matrix is output for the learnable temporal encoder layer, and the matrix can be shared by each temporal encoder layer and data of each dimension, thereby reducing the space occupied by the model and improving the correlation between data of each dimension.
[0037] Then, all the initial time encoder layer outputs are respectively subjected to residual connection, normalization, and feedforward network processing, so that after completing the time layer output processing, multiple time encoder layer outputs of the time encoder layer can be obtained, and the output result of the time encoder layer, that is, the time encoder layer output, is input into the next time encoder layer. Since the structure of each time encoder layer is the same, the multiple time encoder layer outputs are used as the input of the next time encoder layer and the same calculation and processing as the first time encoder layer are performed until the last time encoder layer. The time encoder layer output of the last time encoder layer is used as the time coding sequence.
[0038] S4: constructing a phase conversion layer, inputting all the time coding sequences into the phase conversion layer, and performing multiple tendency feature extractions on all the time coding sequences through the phase conversion layer to obtain multiple feature time series; Furthermore, the purpose of this stage is to extract the tendency features of the time coding sequence, thereby obtaining multiple feature time series to prepare for the subsequent calculation of the variable coding sequence. Furthermore, step S4 further includes: S41: constructing the phase conversion layer including a plurality of feature extraction matrices, and inputting all the time coding sequences into the phase conversion layer; S42: In the stage conversion layer, multiple fusion and tendency feature extraction are performed on all the time coding sequences through multiple feature extraction matrices to obtain multiple feature time series.
[0039] With respect to the above steps, the specific implementation methods in this embodiment are as follows: First, a phase conversion layer including multiple feature extraction matrices is constructed, and multiple different feature extraction matrices are included in the phase conversion layer. Multiple time coding sequences are input into the phase conversion layer, so that in the phase conversion layer, multiple fusion and tendency feature extraction are performed on all time coding sequences through multiple feature extraction matrices, that is, firstly, the time coding sequences in the same dimension are fused to obtain the first Dimensional time coding sequence under , and then calculate the first The first dimension Feature time series : in, For the learnable feature extraction matrix, B is a learnable bias matrix, where each feature extraction matrix can extract variable feature information of the dimensional time coding sequence in different directions, such as the cycle of the data, the change trend, etc. The feature extraction matrix can be obtained under each dimension time coding sequence feature time series, The value of should strike a balance between prediction accuracy and computational efficiency.
[0040] S5: construct a variable encoder of a multi-head attention mechanism, input all the feature time series into the variable encoder, and calculate multiple variable encoding sequences; Furthermore, the purpose of this stage is to calculate the variable feature relationship between the feature time series through the variable encoder and capture the variable feature information between the feature time series. Specifically, step S5 further includes: S51: Establish a variable encoder of a multi-head attention mechanism, wherein the variable encoder includes a plurality of identical variable encoder layers connected in sequence; S52: inputting all the feature time series into the first variable encoder layer in sequence; S53: in the first variable encoder layer, sequentially calculating the variable query vector, the variable key vector and the variable value vector of each of the feature time series, and outputting the variable query vector, the variable key vector and the variable value vector as the initial variable encoder layer; S54: performing variable layer output processing on all the initial variable encoder layer outputs to obtain a plurality of the variable encoder layer outputs; and inputting all the variable encoder layer outputs into the next variable encoder layer; S55: Use the variable encoder layer output of the last variable encoder layer as the variable encoding sequence.
[0041] With respect to the above steps, the specific implementation methods in this embodiment are as follows: First, based on the encoder structure in Transformer, a variable encoder with a multi-head attention mechanism is constructed, and the variable encoder includes multiple identical variable encoder layers connected in sequence. Then all feature time series are input into the first variable encoder layer in sequence. In the first variable encoder layer, the first The first dimension Feature time series In the multi-head attention mechanism Variable query vector , variable key vector and variable value vector : in, For the learnable The variable query vector linear mapping matrix of each head, For the learnable The variable key vector linear mapping matrix of the head, The first The variable value vector linear mapping matrix of each head. The variable query vector, variable key vector and variable value vector of the feature time series in each head are calculated in sequence to obtain the variable query vector, variable key vector and variable value vector of the feature time series and output them as the initial variable encoder layer.
[0042] Then, all the initial variable encoder layer outputs are processed by variable layer output, that is, the variable query vector, variable key vector and variable value vector of each head are calculated to obtain the first variable encoder layer. The first dimension The first characteristic time series Output of the variable attention mechanism : in, () is for softmax transformation, T represents transposition, , represents the variable attention mechanism coefficient, is the number of heads in the multi-head attention mechanism in the variational encoder layer, for After obtaining the variable attention mechanism output of each head, the variable attention mechanism output of each head is fused to obtain the first The first dimension The variable attention mechanism output of feature time series , thus obtaining The first dimension The initial variable encoder layer output of the feature time series : in, The mapping matrix is output for the learnable variable encoder layer, and the matrix can be shared by each variable encoder layer and data of each dimension, thereby reducing the space occupied by the model and improving the correlation between data of each dimension.
[0043] Then, all the initial variable encoder layer outputs are respectively subjected to residual connection, normalization, and feedforward network processing, so that after completing the variable layer output processing, multiple variable encoder layer outputs of the variable encoder layer can be obtained, and the output results of the variable encoder layer are input to the next variable encoder layer. Since the structure of each variable encoder layer is the same, the multiple variable encoder layer outputs are used as the input of the next variable encoder layer and the same processing as the first variable encoder layer is performed until the last variable encoder layer. The variable encoder layer output of the last variable encoder layer is extracted to obtain multiple variable encoding sequences.
[0044] S6: Reorganize and expand all the variable coding sequences to obtain multiple single-dimensional prediction results, obtain multiple target single-dimensional prediction results through multiple single-dimensional non-stationary data and multiple single-dimensional prediction results, and fuse all the target single-dimensional prediction results to obtain the target prediction result.
[0045] Furthermore, the purpose of this stage is to reorganize and project the variable coding sequences to obtain a single-dimensional prediction result, and then obtain multiple target single-dimensional prediction results, and finally obtain the target prediction result through fusion to complete the prediction. Specifically, firstly, all variable coding sequences are reorganized and projected, that is, the variable coding sequences of the same dimension are combined into multiple single-dimensional variable coding sequences. Then, each single-dimensional variable encoding sequence is expanded to obtain multiple single-dimensional expansion results , and then project multiple single-dimensional expansion results to obtain multiple single-dimensional prediction results : in, () for expansion processing, is a learnable linear projection matrix, is the learnable bias projection matrix.
[0046] Then, by calculating each single-dimensional prediction result and the single-dimensional prediction result corresponding to the dimension, the target single-dimensional prediction result of each dimension can be obtained. : Finally, all target single-dimensional prediction results are fused to obtain the target prediction result and complete the prediction.
[0047] The present invention also verifies the effectiveness of the multi-scale correlation time series prediction method. First, the time series prediction task is performed on six public real data sets, namely Weather, Traffic, Electricity, Exchange, ETT and Solar, respectively. The ETT data set consists of four sub-data sets, namely ETTh1, ETTh2, ETTm1 and ETTm2. The detailed description of the data set is shown in Table 1, where the variable dimension is the number of dimensions of the data set, the prediction length is the number of predicted data, for example, 96 means predicting 96 future data based on the time interval, the number of records is the number of data in the data set, the frequency is the time interval for record sampling, and the field is the field to which the data set belongs.
[0048] Table 1 Schematic table of data set contents
[0049] As can be seen from Table 1, the six real data sets used cover time series data from multiple fields, with different characteristics such as variable dimensions, number of records, sampling frequency, etc., which can comprehensively evaluate the performance of the time series prediction model.
[0050] Next, the present invention is used to make predictions and the prediction results are evaluated based on two commonly used evaluation indicators: MSE (Mean Square Error) and MAE (Mean Absolute Error): in, is the predicted length, Indicates Real data, Indicates The experimental results of the time series prediction of the method of the present invention and other methods on real data sets are shown in Table 2, where Avg is the average of the experimental results of the four prediction lengths, the bold results are the data of the best performing model in the same experimental setting, and the underlined results are the data of the second best performing model. is the best number of times the model performs under a certain indicator. It is the number of times the model performs second best under a certain indicator, and the year in brackets is the year when the model was created.
[0051] Table 2 Timing prediction results of the present invention and other models
[0052] It can be seen from Table 2 that compared with the best existing method, the method of the present invention has significantly improved in performance under comprehensive evaluation of various experimental settings. The comparison results fully demonstrate that the method proposed in the present invention has achieved excellent results in time series prediction tasks.
[0053] In addition, an ablation experiment was conducted, that is, some steps in the multi-scale correlation time series prediction method were removed or replaced and time series prediction experiments were conducted. The bold results are the model architectures with the best average performance in the same experimental setting. The results are shown in Table 3: Table 3 Ablation experiment results
[0054] It can be seen from Table 3 that removing or replacing a step will have a negative impact on the final monthly results, which proves that each step in this method plays an indispensable and irreplaceable role.
[0055] The multi-scale associated time series prediction system provided by the present invention is described below. The multi-scale associated time series prediction system described below and the multi-scale associated time series prediction method described above can be referenced to each other.
[0056] Figure 2 The structural diagram of the multi-scale correlation time series prediction system is shown as follows: Figure 2 As shown, the method for performing the multi-scale correlation time series prediction as described above includes: Data stabilization module 100: used to obtain an input sequence, split the input sequence to obtain multiple single-dimensional input sequences, and calculate the stabilized time series and single-dimensional non-stationary data of each single-dimensional input sequence; The time series vector module 200 is used to obtain a position coding enhancement vector, cut each of the smoothed time series to obtain a plurality of sliced time series, embed each of the sliced time series to obtain a plurality of initial time series vectors, and add each of the initial time series vectors to the position coding enhancement vector to obtain a plurality of time series vectors; Time encoder module 300: used to construct a time encoder of a multi-head attention mechanism, input all the time series vectors into the time encoder in sequence, and calculate multiple time code sequences; Phase conversion layer module 400: used to construct a phase conversion layer, input all the time code sequences into the phase conversion layer, and perform multiple tendency feature extractions on all the time code sequences through the phase conversion layer to obtain multiple feature time series; Variable encoder module 500: used to construct a variable encoder of a multi-head attention mechanism, input all the feature time series into the variable encoder, and calculate multiple variable encoding sequences; Target prediction result module 600: used to reorganize and expand the projection processing of all the variable coding sequences to obtain multiple single-dimensional prediction results, obtain multiple target single-dimensional prediction results through multiple single-dimensional non-stationary data and multiple single-dimensional prediction results, and fuse all the target single-dimensional prediction results to obtain the target prediction result.
[0057] on the other hand, Figure 3 An example of a physical structure diagram of an electronic device is shown in FIG. Figure 3 As shown, the electronic device may include: a processor 810, a communication interface 820, a memory 830 and a communication bus 840, wherein the processor 810, the communication interface 820, and the memory 830 communicate with each other through the communication bus 840. The processor 810 may call the logic instructions in the memory 830 to execute the multi-scale correlation time series prediction method, which includes: S1: Obtain an input sequence, split the input sequence to obtain multiple single-dimensional input sequences, and calculate the stationary time series and single-dimensional non-stationary data of each single-dimensional input sequence; S2: Obtain a position coding enhancement vector, cut each of the smoothed time series to obtain multiple slice time series, embed each of the slice time series to obtain multiple initial time series vectors, and add each of the initial time series vectors to the position coding enhancement vector to obtain multiple time series vectors; S3: construct a time encoder of a multi-head attention mechanism, input all the time series vectors into the time encoder in sequence, and calculate multiple time code sequences; S4: constructing a phase conversion layer, inputting all the time coding sequences into the phase conversion layer, and performing multiple tendency feature extractions on all the time coding sequences through the phase conversion layer to obtain multiple feature time series; S5: construct a variable encoder of a multi-head attention mechanism, input all the feature time series into the variable encoder, and calculate multiple variable encoding sequences; S6: Reorganize and expand all the variable coding sequences to obtain multiple single-dimensional prediction results, obtain multiple target single-dimensional prediction results through multiple single-dimensional non-stationary data and multiple single-dimensional prediction results, and fuse all the target single-dimensional prediction results to obtain the target prediction result.
[0058] In addition, the logic instructions in the above-mentioned memory 830 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on this understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art or the part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.
[0059] On the other hand, the present invention further provides a computer program product, the computer program product comprising a computer program stored on a non-transitory computer-readable storage medium, the computer program comprising program instructions, when the program instructions are executed by a computer, the computer can execute the multi-scale correlation time series prediction method provided by the above methods, the method comprising: S1: Obtain an input sequence, split the input sequence to obtain multiple single-dimensional input sequences, and calculate the stationary time series and single-dimensional non-stationary data of each single-dimensional input sequence; S2: Obtain a position coding enhancement vector, cut each of the smoothed time series to obtain multiple slice time series, embed each of the slice time series to obtain multiple initial time series vectors, and add each of the initial time series vectors to the position coding enhancement vector to obtain multiple time series vectors; S3: construct a time encoder of a multi-head attention mechanism, input all the time series vectors into the time encoder in sequence, and calculate multiple time code sequences; S4: constructing a phase conversion layer, inputting all the time coding sequences into the phase conversion layer, and performing multiple tendency feature extractions on all the time coding sequences through the phase conversion layer to obtain multiple feature time series; S5: construct a variable encoder of a multi-head attention mechanism, input all the feature time series into the variable encoder, and calculate multiple variable encoding sequences; S6: Reorganize and expand all the variable coding sequences to obtain multiple single-dimensional prediction results, obtain multiple target single-dimensional prediction results through multiple single-dimensional non-stationary data and multiple single-dimensional prediction results, and fuse all the target single-dimensional prediction results to obtain the target prediction result.
[0060] In another aspect, the present invention further provides a non-transitory computer-readable storage medium having a computer program stored thereon, which is implemented when the computer program is executed by a processor to perform the multi-scale correlation time series prediction method provided by the above methods, the method comprising: S1: Obtain an input sequence, split the input sequence to obtain multiple single-dimensional input sequences, and calculate the stationary time series and single-dimensional non-stationary data of each single-dimensional input sequence; S2: Obtain a position coding enhancement vector, cut each of the smoothed time series to obtain multiple slice time series, embed each of the slice time series to obtain multiple initial time series vectors, and add each of the initial time series vectors to the position coding enhancement vector to obtain multiple time series vectors; S3: construct a time encoder of a multi-head attention mechanism, input all the time series vectors into the time encoder in sequence, and calculate multiple time code sequences; S4: constructing a phase conversion layer, inputting all the time coding sequences into the phase conversion layer, and performing multiple tendency feature extractions on all the time coding sequences through the phase conversion layer to obtain multiple feature time series; S5: construct a variable encoder of a multi-head attention mechanism, input all the feature time series into the variable encoder, and calculate multiple variable encoding sequences; S6: Reorganize and expand all the variable coding sequences to obtain multiple single-dimensional prediction results, obtain multiple target single-dimensional prediction results through multiple single-dimensional non-stationary data and multiple single-dimensional prediction results, and fuse all the target single-dimensional prediction results to obtain the target prediction result.
[0061] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.
Claims
1. A multi-scale correlation time series prediction method, characterized in that: include: S1: Obtain an input sequence, split the input sequence to obtain multiple single-dimensional input sequences, and calculate the stationary time series and single-dimensional non-stationary data of each single-dimensional input sequence; S2: Obtain a position coding enhancement vector, cut each of the smoothed time series to obtain multiple slice time series, embed each of the slice time series to obtain multiple initial time series vectors, and add each of the initial time series vectors to the position coding enhancement vector to obtain multiple time series vectors; S3: construct a time encoder of a multi-head attention mechanism, input all the time series vectors into the time encoder in sequence, and calculate multiple time code sequences; S4: constructing a phase conversion layer, inputting all the time coding sequences into the phase conversion layer, and performing multiple tendency feature extractions on all the time coding sequences through the phase conversion layer to obtain multiple feature time series; S5: construct a variable encoder of a multi-head attention mechanism, input all the feature time series into the variable encoder, and calculate multiple variable encoding sequences; S6: Reorganize and expand all the variable coding sequences to obtain multiple single-dimensional prediction results, obtain multiple target single-dimensional prediction results through multiple single-dimensional non-stationary data and multiple single-dimensional prediction results, and fuse all the target single-dimensional prediction results to obtain the target prediction result.
2. The multi-scale correlation time series prediction method according to claim 1, characterized in that: Step S1 specifically includes: S11: Acquire an input sequence, determine the data dimension of the data in the input sequence, and split the input sequence according to the data dimension to obtain a plurality of single-dimensional input sequences; S12: Calculate the dimensional data mean and dimensional data standard deviation of each of the single-dimensional input sequences, calculate the stationary time series of each of the single-dimensional input sequences through the dimensional data mean and the dimensional data standard deviation, and use the dimensional data mean and the dimensional data standard deviation as the single-dimensional non-stationary data of the single-dimensional input sequence.
3. The multi-scale correlation time series prediction method according to claim 1, characterized in that: Step S3 specifically includes: S31: Establish a time encoder of a multi-head attention mechanism, wherein the time encoder includes a plurality of identical time encoder layers connected in sequence; S32: input all the time series vectors into the first time encoder layer in sequence; S33: in the first time encoder layer, sequentially calculating a time query vector, a time key vector and a time value vector of each time series vector, and outputting the time query vector, the time key vector and the time value vector as an initial time encoder layer; S34: performing time layer output processing on all the initial time encoder layer outputs to obtain a plurality of the time encoder layer outputs, and inputting all the time encoder layer outputs into the next time encoder layer; S35: Output the time encoder layer of the last time encoder layer as the time encoding sequence.
4. The multi-scale correlation time series prediction method according to claim 1, characterized in that: Step S4 specifically includes: S41: constructing the phase conversion layer including a plurality of feature extraction matrices, and inputting all the time coding sequences into the phase conversion layer; S42: In the stage conversion layer, multiple fusion and tendency feature extraction are performed on all the time coding sequences through multiple feature extraction matrices to obtain multiple feature time series.
5. The multi-scale correlation time series prediction method according to claim 1, characterized in that: Step S5 specifically includes: S51: Establish a variable encoder of a multi-head attention mechanism, wherein the variable encoder includes a plurality of identical variable encoder layers connected in sequence; S52: inputting all the feature time series into the first variable encoder layer in sequence; S53: in the first variable encoder layer, sequentially calculating the variable query vector, the variable key vector and the variable value vector of each of the feature time series, and outputting the variable query vector, the variable key vector and the variable value vector as the initial variable encoder layer; S54: performing variable layer output processing on all the initial variable encoder layer outputs to obtain a plurality of the variable encoder layer outputs; and inputting all the variable encoder layer outputs into the next variable encoder layer; S55: Use the variable encoder layer output of the last variable encoder layer as the variable encoding sequence.
6. The multi-scale correlation time series prediction method according to claim 1, characterized in that: In step S2, before cutting the smoothed time series, the slice interval S is first determined, and then the last value of the smoothed time series is repeated S times to fill the tail of each smoothed time series.
7. A multi-scale correlation time series prediction system, used to execute the multi-scale correlation time series prediction method according to any one of claims 1 to 6, characterized in that: include: Data stabilization module: used to obtain an input sequence, split the input sequence to obtain multiple single-dimensional input sequences, and calculate the stabilized time series and single-dimensional non-stationary data of each single-dimensional input sequence; Time series vector module: used to obtain the position coding enhancement vector, cut each of the smoothed time series to obtain multiple sliced time series, embed each of the sliced time series to obtain multiple initial time series vectors, and add each of the initial time series vectors to the position coding enhancement vector to obtain multiple time series vectors; Time encoder module: used to construct a time encoder of a multi-head attention mechanism, input all the time series vectors into the time encoder in sequence, and calculate multiple time code sequences; Phase conversion layer module: used to construct a phase conversion layer, input all the time code sequences into the phase conversion layer, and perform multiple tendency feature extractions on all the time code sequences through the phase conversion layer to obtain multiple feature time series; Variable encoder module: used to construct a variable encoder of a multi-head attention mechanism, input all the feature time series into the variable encoder, and calculate multiple variable encoding sequences; Target prediction result module: used to reorganize and expand the projection processing of all the variable coding sequences to obtain multiple single-dimensional prediction results, obtain multiple target single-dimensional prediction results through multiple single-dimensional non-stationary data and multiple single-dimensional prediction results, and fuse all the target single-dimensional prediction results to obtain the target prediction result.
8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the steps of the multi-scale correlation time series prediction method according to any one of claims 1 to 6 are implemented.
9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the multi-scale correlation time series prediction method according to any one of claims 1 to 6 are implemented.
10. A computer program product, the computer program product comprising a computer program stored on a non-transitory computer-readable storage medium, the computer program comprising program instructions, characterized in that: When the program instructions are executed by a computer, the computer can perform the steps of the multi-scale correlation time series prediction method as described in any one of claims 1 to 6.