Long-term time sequence prediction method and device based on sequence decomposition and multi-scale attention mechanism

Abstract

The invention relates to a long-term time sequence prediction method, device and system based on sequence decomposition and a multi-scale attention mechanism, and a storage medium, relates to the technical field of artificial intelligence, and at least solves the problems of high fluctuation and intermittency of data in a long-term prediction task. The method comprises the following steps: performing sequence decomposition on original data to obtain a trend component and a season component; the trend component represents the trend of the original data in a long time span, and the season component represents the periodic change trend of the original data influenced by the environment; predicting the trend component to obtain a first prediction result, and predicting the season component to obtain a second prediction result; and fusing the first prediction result and the second prediction result to obtain a target preset result.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and in particular to a method and apparatus for long-term time series prediction based on sequence decomposition and multi-scale attention mechanism. Background Technology

[0002] Time series forecasting has been widely applied in energy consumption, transportation and economic planning, and weather and disease spread forecasting. In these practical applications, a pressing need is to extend the forecast timeframe to the distant future, which is highly significant for long-term planning and early warning. Long-term time series forecasting is characterized by the large length of the predicted time series. Recent deep forecasting models have made significant progress, particularly Transformer-based models. Benefiting from the self-attention mechanism, Transformers have a significant advantage in modeling long-term dependencies in sequence data, enabling the construction of more robust models. However, the forecasting task is extremely challenging in the long-term context. First, directly discovering temporal dependencies from long-term time series is unreliable, as dependencies may be masked by entangled temporal patterns. Second, due to the quadratic complexity of the sequence length, standard Transformers with self-attention mechanisms are computationally infeasible for long-term forecasting.

[0003] To address the quadratic complexity of sequence length in Transformers, common techniques involve sparsifying the self-attention mechanism, resulting in various improved attention mechanisms. To enhance prediction accuracy, some methods focus on improving the Transformer structure and designing the input data. While these methods have achieved some success, their prediction accuracy often falls short when dealing with intermittent and highly volatile data. Therefore, there is an urgent need to find a long-term time series forecasting method capable of handling frequently fluctuating data. Summary of the Invention

[0004] This invention provides a method and apparatus for long-term time series forecasting based on sequence decomposition and multi-scale attention mechanisms, to at least address the problems of high volatility and intermittency in data during long-term forecasting tasks. The technical solution of this invention is as follows:

[0005] According to a first aspect of the present invention, a long-term time series prediction method based on sequence decomposition and multi-scale attention mechanism is provided. The method includes: performing sequence decomposition on the original data to obtain a trend component and a seasonal component; the trend component represents the trend of the original data over a long time span, and the seasonal component represents the periodic change trend of the original data under the influence of the environment; predicting the trend component to obtain a first prediction result, and predicting the seasonal component separately to obtain a second prediction result; and fusing the first prediction result and the second prediction result to obtain a target preset result.

[0006] In one implementation, the original data is decomposed into a sequence to obtain a trend component and a seasonal component, including: performing a padding operation on the original sequence to obtain a padded original sequence; using a moving average function to process the padded original sequence to obtain the trend component; and subtracting the trend component from the original sequence to obtain the seasonal component.

[0007] Another implementation involves predicting the trend component to obtain a first prediction result; this includes using linear regression to predict the trend component and obtain a first prediction result.

[0008] In another implementation, the seasonal components are predicted, and the second prediction result includes: temporal encoding of the seasonal components to obtain a first sequence, the first sequence including temporal features; and constructing a multi-scale attention mechanism by processing a key matrix within a preset scale range according to each query matrix each time; constructing a first multi-scale sequence based on the first sequence; the scale of each scale sequence in the first multi-scale sequence is positively correlated with the position of the scale sequence; processing the first multi-scale sequence according to the multi-scale attention mechanism to obtain a second sequence; constructing a second multi-scale sequence based on the second sequence; encoding the variable dimensions of the second multi-scale sequence to obtain a third sequence; processing the third sequence according to the multi-scale attention mechanism to obtain a fourth sequence; and performing forward propagation and residual connection processing on the fourth sequence to obtain the second prediction result.

[0009] In another implementation, a first multi-scale sequence is constructed based on the first sequence, including: taking the first sequence as the first scale, performing convolution operations on the convolution results of the previous scale one by one until the scale of the obtained convolution result is greater than or equal to a preset scale or until a preset number of convolutions is reached, so as to obtain multiple first convolution results corresponding to different scales; and connecting the first sequence and multiple first convolution results to obtain the first multi-scale sequence.

[0010] In another implementation, a second multi-scale sequence is constructed based on the second sequence; including: taking the second sequence as the first scale, performing convolution operations on the convolution results of the previous scale one by one until the scale of the obtained convolution result is greater than or equal to a preset scale or until a preset number of convolutions is reached, so as to obtain multiple second convolution results corresponding to different scales; and concatenating the second sequence and multiple second convolution results to obtain the second multi-scale sequence.

[0011] According to a second aspect of the present invention, a long-term time series prediction device based on sequence decomposition and multi-scale attention mechanism is provided. The device includes: a decomposition unit for performing sequence decomposition on the original data to obtain a trend component and a seasonal component; a prediction unit for predicting the trend component to obtain a first prediction result, and predicting the seasonal component to obtain a second prediction result; and a fusion unit for fusing the first prediction result and the second prediction result to obtain a target preset result.

[0012] In one implementation, the decomposition unit is used to: perform a padding operation on the original sequence to obtain the padded original sequence; use a moving average function to process the padded original sequence to obtain the trend component; and subtract the trend component from the original sequence to obtain the seasonal component.

[0013] In another implementation, the prediction unit is specifically used to: predict the trend component using linear regression to obtain the first prediction result.

[0014] In another implementation, the prediction unit is specifically used for: temporally encoding the seasonal components to obtain a first sequence, the first sequence including temporal features; and, processing the key matrix within a preset scale range according to each query matrix to construct a multi-scale attention mechanism; constructing a first multi-scale sequence based on the first sequence; the scale of each scale sequence in the first multi-scale sequence is positively correlated with the position of the scale sequence; processing the first multi-scale sequence according to the multi-scale attention mechanism to obtain a second sequence; constructing a second multi-scale sequence based on the second sequence; encoding the variable dimensions of the second multi-scale sequence to obtain a third sequence; processing the third sequence according to the multi-scale attention mechanism to obtain a fourth sequence; and performing forward propagation and residual connection processing on the fourth sequence to obtain a second prediction result.

[0015] In another implementation, the prediction unit is specifically used to: take the first sequence as the first scale, and successively perform convolution operations on the convolution results of the previous scale until the scale of the obtained convolution result is greater than or equal to the preset scale or until the preset number of convolutions is reached, so as to obtain multiple first convolution results corresponding to different scales; and connect the first sequence and multiple first convolution results to obtain the first multi-scale sequence.

[0016] In another implementation, the second multi-scale sequence prediction unit is constructed based on the second sequence and is specifically used to: take the second sequence as the first scale, perform convolution operation on the convolution result of the previous scale one by one until the scale of the obtained convolution result is greater than or equal to the preset scale or until the preset number of convolutions is reached, so as to obtain multiple second convolution results corresponding to different scales; connect the second sequence and multiple second convolution results to obtain the second multi-scale sequence.

[0017] According to a third aspect of the present invention, a long-term time series prediction system based on sequence decomposition and multi-scale attention mechanism is provided, the system being configured to perform a long-term time series prediction method based on sequence decomposition and multi-scale attention mechanism as described in the first aspect and any possible implementation thereof.

[0018] According to a fourth aspect of the present invention, a computer-readable storage medium is provided, on which instructions are stored, such that when the instructions in the computer-readable storage medium are executed by a processor of an electronic device, the electronic device is able to perform a long-term time series prediction method based on sequence decomposition and multi-scale attention mechanism, as described in the first aspect and any possible implementation thereof.

[0019] According to a fifth aspect of the present disclosure, a computer program product is provided, the computer program product including computer instructions, which, when executed on an electronic device, cause the electronic device to perform the long-term time series prediction method based on sequence decomposition and multi-scale attention mechanism described in the first aspect and any possible implementation thereof.

[0020] The technical solution provided by the embodiments of this application brings at least the following beneficial effects: The original data is decomposed into two data trend dimensions: a trend component and a seasonal component. The original data is then characterized by these two components. The trend component and the seasonal component are predicted separately, and the prediction results are fused to obtain the final prediction result. In the above data prediction process, compared to predicting directly based on the original data, predicting using these two components effectively reduces the high volatility and intermittency of the data, thereby improving the accuracy of long-term data prediction.

[0021] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description

[0022] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure, and are not intended to unduly limit this disclosure.

[0023] Figure 1This is a flowchart illustrating a long-term time series prediction method based on sequence decomposition and multi-scale attention mechanism according to an exemplary embodiment;

[0024] Figure 2 This is a schematic diagram illustrating a seasonal component prediction process according to an exemplary embodiment;

[0025] Figure 3 This is a schematic diagram illustrating a long-term time series forecasting process according to an exemplary embodiment;

[0026] Figure 4 This is a schematic diagram illustrating a convolution operation process according to an exemplary embodiment;

[0027] Figure 5 This is a graphical schematic diagram illustrating multi-scale attention according to an exemplary embodiment;

[0028] Figure 6 This is a block diagram illustrating a long-term time series prediction device based on sequence decomposition and multi-scale attention mechanism according to an exemplary embodiment. Detailed Implementation

[0029] To enable those skilled in the art to better understand the technical solutions of this disclosure, the technical solutions in the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings.

[0030] It should be noted that the terms "first," "second," etc., used in the specification, claims, and accompanying drawings of this disclosure are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this disclosure described herein can be implemented in orders other than those illustrated or described herein. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this disclosure. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this disclosure as detailed in the appended claims.

[0031] Before providing a detailed introduction to the long-term time series prediction method based on sequence decomposition and multi-scale attention mechanism provided in the embodiments of this application, let's first briefly introduce the application scenarios involved in the embodiments of this application.

[0032] Time series forecasting has been widely applied in energy consumption, transportation and economic planning, and weather and disease spread forecasting. In these practical applications, a pressing need is to extend the forecast timeframe to the distant future, which is highly significant for long-term planning and early warning. Long-term time series forecasting is characterized by the large length of the predicted time series. Recent deep forecasting models have made significant progress, particularly Transformer-based models. Benefiting from the self-attention mechanism, Transformers have a significant advantage in modeling long-term dependencies in sequence data, enabling the construction of more robust models. However, the forecasting task is extremely challenging in the long-term context. First, directly discovering temporal dependencies from long-term time series is unreliable, as dependencies may be masked by entangled temporal patterns. Second, due to the quadratic complexity of the sequence length, standard Transformers with self-attention mechanisms are computationally infeasible for long-term forecasting.

[0033] To address the quadratic complexity of sequence length in Transformers, common techniques involve sparsifying the self-attention mechanism, leading to various improved attention mechanisms. To enhance prediction accuracy, some methods focus on improving the Transformer structure and designing the input data. While these methods have achieved some success, their prediction accuracy often falls short when dealing with intermittent and highly volatile data. Therefore, there is an urgent need to find a long-term time series forecasting method capable of handling frequently fluctuating data.

[0034] To address the aforementioned issues, this application proposes a long-term time series prediction method based on sequence decomposition and multi-scale attention mechanism.

[0035] For ease of understanding, the long-term time series prediction method based on sequence decomposition and multi-scale attention mechanism provided in this application will be described in detail below with reference to the accompanying drawings. This long-term time series prediction method based on sequence decomposition and multi-scale attention mechanism is applied to the aforementioned long-term time series prediction system based on sequence decomposition and multi-scale attention mechanism.

[0036] Figure 1 This is a flowchart illustrating a long-term time series prediction method based on sequence decomposition and multi-scale attention mechanism according to an exemplary embodiment, such as... Figure 1 As shown, the long-term time series prediction method based on sequence decomposition and multi-scale attention mechanism includes the following steps.

[0037] S11 performs sequence decomposition on the original data to obtain trend components and seasonal components.

[0038] The trend component represents the trend of the original data over a long period of time, while the seasonal component represents the periodic change trend of the original data due to environmental influences.

[0039] S12, predict the trend component to obtain the first prediction result, and predict the seasonal component to obtain the second prediction result.

[0040] S13, the first prediction result and the second prediction result are merged to obtain the target preset result.

[0041] Through the above implementation method, the original data is decomposed into two data trend dimensions: a trend component and a seasonal component. These two components represent the original data, and predictions are made separately for each component. The prediction results are then fused to obtain the final prediction result. Compared to predicting directly based on the original data, this method of predicting using two components effectively reduces the high volatility and intermittency of the data, thereby improving the accuracy of long-term data predictions.

[0042] As a refinement and extension of the specific implementation of the above embodiments, in order to fully explain the specific implementation process of this embodiment, the above implementation is further explained through the following implementation steps.

[0043] Before proceeding with the specifics, we will define the problem addressed in this application, focusing on long-term forecasting tasks for time series data. Let x... t ∈R m (m≥1) represents data at time t, where m represents the number of relevant variables. When m=1, there is only one variable feature. Therefore, the sequence data over the total duration T can be represented as X. 1:T ={x1, x1, ... x T}∈R T×m Furthermore, let the resolution of the data (i.e., the sampling interval) be τ, such as 10 minutes, 15 minutes, or 1 hour.

[0044] Based on the above definition, the long-term time series prediction problem can be formulated as: given k steps of historical data from observations... Predict the future L steps Y by training the forward propagation function forecast(*). t+1:t+L Right now Furthermore, L*τ must be greater than one day for it to be defined as a long-term forecast.

[0045] As one implementation method, the original data in step S11 is decomposed into sequences in the following manner.

[0046] First, the original sequence is padded to obtain the padded original sequence.

[0047] Secondly, a moving average function is used to process the padded original sequence pairs to obtain the trend component.

[0048] Third, the seasonal component is obtained by subtracting the trend component from the original sequence.

[0049] Specifically, the decomposition of the original sequence is achieved by using a moving average to smooth the sequence and obtain the trend component. Then, the seasonal component is obtained by subtracting the trend component from the original sequence. The specific representation is shown in formulas (1) and (2).

[0050]

[0051] in, X t-k+1:t For historical observation data sequences, For seasonal portions, For the trend component. Padding(*) is the padding function, and AvgPool(*) is the moving average function to keep the sequence length constant.

[0052] As one implementation method, such as Figure 2 The implementation process of the above time series forecasting will be further explained using the implementation process shown as an example.

[0053] Historical observation data sequences are used as the raw sequence and input into the sequence decomposition module to obtain seasonal and predicted components. The seasonal components are then input into the seasonal prediction module, and the trend components are input into the trend prediction module, correspondingly outputting the first prediction result for the trend components. Second prediction results for seasonal components Then, the first and second prediction results output by the two modules are added together to obtain the target prediction result.

[0054] As one implementation method, the specific implementation process of predicting the trend component in S12 above to obtain the first prediction result is as follows: the trend component is predicted by using linear regression to obtain the first prediction result.

[0055] Specifically, for the trend component, linear regression is used to predict it, and this linear regression can be expressed as:

[0056]

[0057] Where W∈R L*k The weight matrix, W, is initially set as follows: This holds true for all i and j.

[0058] As one implementation method, the specific implementation process of predicting the seasonal component in S12 above is as follows.

[0059] First, the seasonal components are time-encoded to obtain the first sequence.

[0060] The first sequence includes time features.

[0061] By encoding the obtained seasonal components over time, it becomes easier to extract time-dimensional information in the future.

[0062] Secondly, a multi-scale attention mechanism is constructed by processing key matrices within a preset scale range for each query matrix.

[0063] Third, based on the first sequence, a first multi-scale sequence is constructed.

[0064] In the first multiscale sequence, the scale of each scale sequence is positively correlated with the position of the scale sequence.

[0065] Fourth, the first multi-scale sequence is processed according to the multi-scale attention mechanism to obtain the second sequence.

[0066] Fifth, based on the second sequence, a second multi-scale sequence is constructed.

[0067] Sixth, the second multi-scale sequence is encoded with variable dimensions to obtain the third sequence.

[0068] Seventh, the third sequence is processed according to the multi-scale attention mechanism to obtain the fourth sequence.

[0069] Eighth, forward propagation and residual connection processing are performed on the fourth sequence to obtain the second prediction result.

[0070] As one implementation method, a first multi-scale sequence is constructed based on a first sequence, including: taking the first sequence as the first scale, performing convolution operations on the convolution results of the previous scale one by one until the scale of the obtained convolution results is greater than or equal to a preset scale or until a preset number of convolutions is reached, so as to obtain multiple first convolution results corresponding to different scales; and connecting the first sequence and multiple first convolution results to obtain the first multi-scale sequence.

[0071] As another implementation method, a second multi-scale sequence is constructed based on the second sequence; including:

[0072] Using the second sequence as the first scale, convolution operations are performed on the convolution results of the previous scale one by one until the scale of the obtained convolution result is greater than or equal to the preset scale or until the preset number of convolutions is reached, so as to obtain multiple second convolution results corresponding to different scales.

[0073] The second sequence is concatenated with multiple second convolution results to obtain the second multi-scale sequence.

[0074] like Figure 3As shown, a two-stage multi-scale attention mechanism is used to predict seasonal components.

[0075] First, in order to extract multi-scale information, it is necessary to construct multi-scale sequences.

[0076] like Figure 4 The convolution operation shown uses convolution to construct multi-scale sequences.

[0077] Specifically, a convolution with a fixed kernel size and stride is applied to either the first or third sequence, followed by a convolution of the previous results, and so on. Each convolution operation introduces a new scale. Then, the first or third sequence is concatenated with the convolution results to obtain a multi-scale sequence. That is: X i =Conv(X i-1 ), where X i-1 It is the sequence corresponding to scale i-1; Conv is the convolution operation function. For the sequence X0 at the first scale, it is either the first sequence or the third sequence.

[0078] X multi = [X0, X1, ..., X n ]

[0079] Where [·] represents the join operation, X multi Multiscale sequences, X n It is the highest-scale sequence.

[0080] Once the multi-scale sequence is ready, the proposed multi-scale attention mechanism can be applied to the multi-scale sequence to extract multi-scale information.

[0081] Secondly, before delving into the multi-scale attention mechanism proposed in this application, it is necessary to understand the full attention mechanism in related technologies. Let X represent the input of the full attention mechanism, and Y represent the output. Initially, X is linearly transformed into three separate matrices: query Q = XW Q Key K = XW K Sum of values ​​V = XW V ,in For the i-th row q in q, it can focus on any row (i.e., the key) within k. Therefore, the output y i It can be described as:

[0082]

[0083] in, Let represent the transpose of the j-th row in k. Clearly, the computational cost is quadratic with the sequence length, making the full attention mechanism impractical for long sequences.

[0084] In contrast, the multi-scale attention mechanism proposed in this application focuses on a subset of dot product computation.

[0085] Specifically, each query q only processes nearby keys k. It's important to note that in this case, "nearby" does not refer to temporal proximity, but rather to a local range within the same scale, including several keypoints before and after the current scale, as well as keypoints from adjacent higher and lower scales. Therefore, the multi-scale attention mechanism can be described as:

[0086]

[0087] in This indicates that the i-th node needs to be scaled. (i.e., the subset of nodes computed in the "nearby" area).

[0088] This approach significantly reduces the complexity of the attention mechanism, improves computational efficiency, and maintains accuracy.

[0089] like Figure 5 As shown, a graphical representation of multi-scale attention is provided.

[0090] This indicates that, compared to the full attention mechanism, the multi-scale attention mechanism in this application not only reduces the number of dot product calculations, but also effectively captures multiple scales.

[0091] Secondly, variable dimension encoding is used to facilitate subsequent exploration of relationships between variables.

[0092] Secondly, the multi-scale attention mechanism is used twice to apply it to the variable dimension and explore the correlation between variables.

[0093] Finally, forward propagation and residual connections are used to predict the output results and avoid performance degradation.

[0094] In the above embodiments, decomposing the original time series data into trend components and seasonal components is a crucial foundation for the accurate prediction achieved in this invention. In practice, the moving average method is used to process the original series.

[0095] Specifically, the original sequence is first padded to ensure its length remains constant throughout the processing. This is a crucial step for maintaining data integrity and the consistency of subsequent analysis. After padding, a moving average function is used to calculate the trend component. The moving average method effectively smooths the data and highlights its long-term trends, allowing the trend component to clearly show the data's trajectory over a longer time span. Subsequently, the trend component is subtracted from the original sequence to obtain the seasonal component. The seasonal component contains the periodic variations in the data, which may be influenced by various factors such as seasonal changes, differences between weekdays and weekends, and holidays, exhibiting complex patterns.

[0096] Furthermore, a two-stage multi-scale attention mechanism is employed to improve the prediction accuracy for seasonal component forecasting.

[0097] First, a multi-scale structure is constructed by extracting multi-scale information through a series of carefully designed convolutional operations. Specifically, a convolution with a fixed kernel size and stride is applied to either the first or third sequence, introducing a new scale with each convolution operation. Using the first or third sequence as the first scale, progressive convolutions are performed to obtain higher scales. Then, the first or third sequence is concatenated with each convolution result to form a multi-scale sequence. This multi-scale sequence can capture feature changes in data at different temporal granularities; small scales can capture short-term fluctuations, while large scales reflect long-term periodic changes, greatly enriching the feature representation of the data. After constructing the multi-scale sequence, a multi-scale attention mechanism is used to mine the information within it.

[0098] Unlike traditional full attention mechanisms, the multi-scale attention mechanism of this invention processes only nearby keys for each query. Here, "nearby" does not refer to temporal proximity, but rather to a local range within the same scale, as well as key points from adjacent higher and lower scales. This approach significantly reduces the computational complexity of the attention mechanism, improving computational efficiency while more effectively capturing multi-scale information and enhancing the model's ability to handle complex nonlinear relationships. Subsequently, to uncover correlations between variables, the multi-scale sequences are dimensionally encoded, and the multi-scale attention mechanism is used again to process the variable dimensions, further exploring the potential connections between variables. Finally, the prediction results are output through forward propagation and residual connections. The introduction of residual connections effectively prevents model performance degradation, ensuring the stability and reliability of the predictions.

[0099] To achieve the above functions, the long-term time series prediction device based on sequence decomposition and multi-scale attention mechanisms includes hardware structures and / or software modules corresponding to the execution of each function. Those skilled in the art will readily recognize that, based on the algorithmic steps of the examples described in conjunction with the embodiments disclosed herein, this application can be implemented in hardware or a combination of hardware and computer software. Whether a function is executed in hardware or by computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0100] This disclosure also provides an embodiment such as Figure 6 The long-term time series prediction device shown is based on sequence decomposition and multi-scale attention mechanism. The device includes: a decomposition unit 61 for performing sequence decomposition on the original data to obtain trend components and seasonal components; a prediction unit 62 for predicting the trend components to obtain a first prediction result, and predicting the seasonal components to obtain a second prediction result; and a fusion unit 63 for fusing the first prediction result and the second prediction result to obtain a target preset result.

[0101] In one embodiment, the decomposition unit 61 is used to: perform a filling operation on the original sequence to obtain a filled original sequence; use a moving average function to process the filled original sequence to obtain a trend component; and subtract the trend component from the original sequence to obtain a seasonal component.

[0102] In another implementation, the prediction unit 62 is specifically used to: predict the trend component using linear regression to obtain a first prediction result.

[0103] In another implementation, the prediction unit 62 is specifically used for: temporally encoding the seasonal components to obtain a first sequence, the first sequence including temporal features; and, processing the key matrix within a preset scale range according to each query matrix to construct a multi-scale attention mechanism; constructing a first multi-scale sequence based on the first sequence; the scale of each scale sequence in the first multi-scale sequence is positively correlated with the position of the scale sequence; processing the first multi-scale sequence according to the multi-scale attention mechanism to obtain a second sequence; constructing a second multi-scale sequence based on the second sequence; encoding the variable dimensions of the second multi-scale sequence to obtain a third sequence; processing the third sequence according to the multi-scale attention mechanism to obtain a fourth sequence; and performing forward propagation and residual connection processing on the fourth sequence to obtain a second prediction result.

[0104] In another embodiment, the prediction unit 62 is specifically used to: take the first sequence as the first scale, and successively perform convolution operation on the convolution result of the previous scale until the scale of the obtained convolution result is greater than or equal to the preset scale or until the preset number of convolutions is reached, so as to obtain multiple first convolution results corresponding to different scales; and connect the first sequence and multiple first convolution results to obtain a first multi-scale sequence.

[0105] In another embodiment, the prediction unit 62 is specifically used to: take the second sequence as the first scale, successively perform convolution operation on the convolution result of the previous scale until the scale of the obtained convolution result is greater than or equal to the preset scale or until the preset number of convolutions is reached, so as to obtain multiple second convolution results corresponding to different scales; and connect the second sequence and multiple second convolution results to obtain a second multi-scale sequence.

[0106] Regarding the apparatus in the above embodiments, the specific manner in which each unit module performs its operations has been described in detail in the embodiments related to the method, and will not be elaborated upon here.

[0107] This application also provides a computer-readable storage medium. When the instructions in the computer-readable storage medium are executed by the processor of a long-term time series prediction device or apparatus based on sequence decomposition and multi-scale attention mechanism, the long-term time series prediction device or apparatus based on sequence decomposition and multi-scale attention mechanism is able to perform the long-term time series prediction method based on sequence decomposition and multi-scale attention mechanism as described in any of the above possible embodiments, and can achieve the same technical effect. To avoid repetition, it will not be described again here.

[0108] This application also provides a computer program product, including a computer program or instructions, which are executed by a processor as described in any of the possible implementations above, for a long-term time series prediction method based on sequence decomposition and multi-scale attention mechanisms. This achieves the same technical effect, and to avoid repetition, it will not be described again here.

[0109] Other embodiments of this application will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of this application that follow the general principles of this application and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this application are indicated by the following claims.

[0110] It should be understood that this application is not limited to the precise structure described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this application is limited only by the appended claims.

Claims

1. A long-term time series prediction method based on sequence decomposition and multi-scale attention mechanism, characterized in that, The method includes: The original data is decomposed into a sequence to obtain a trend component and a seasonal component. The trend component represents the trend of the original data over a long period of time, and the seasonal component represents the periodic change trend of the original data under the influence of the environment. The trend component is predicted to obtain a first prediction result, and the seasonal component is predicted to obtain a second prediction result. The first prediction result and the second prediction result are fused together to obtain the target preset result.

2. The method according to claim 1, characterized in that, The step of performing sequence decomposition on the original data to obtain trend components and seasonal components includes: The original sequence is padded to obtain the padded original sequence; The trend component is obtained by processing the padded original sequence pairs using a moving average function. The seasonal component is obtained by subtracting the trend component from the original sequence.

3. The method according to claim 1, characterized in that, The prediction of the trend component to obtain a first prediction result includes: The trend component is predicted using linear regression to obtain the first prediction result.

4. The method according to any one of claims 1 to 3, characterized in that, The prediction of seasonal components, the second prediction result includes: The seasonal components are time-encoded to obtain a first sequence, which includes time features; and a multi-scale attention mechanism is constructed by processing a key matrix within a preset scale range according to each query matrix. Based on the first sequence, a first multi-scale sequence is constructed; the scale of each scale sequence in the first multi-scale sequence is positively correlated with the position of the scale sequence. The first multi-scale sequence is processed according to the multi-scale attention mechanism to obtain the second sequence; Based on the second sequence, a second multi-scale sequence is constructed; The second multi-scale sequence is encoded by variable dimension to obtain the third sequence; The third sequence is processed according to the multi-scale attention mechanism to obtain the fourth sequence; The fourth sequence is subjected to forward propagation and residual connection processing to obtain the second prediction result.

5. The method according to claim 4, characterized in that, The construction of the first multi-scale sequence based on the first sequence includes: Using the first sequence as the first scale, convolution operations are performed on the convolution results of the previous scale one by one until the scale of the obtained convolution result is greater than or equal to the preset scale or until the preset number of convolutions is reached, so as to obtain multiple first convolution results corresponding to different scales. The first sequence and the plurality of first convolution results are concatenated to obtain the first multi-scale sequence.

6. The method according to claim 4, characterized in that, The construction of a second multi-scale sequence based on the second sequence includes: Using the second sequence as the first scale, convolution operations are performed on the convolution results of the previous scale one by one until the scale of the obtained convolution result is greater than or equal to the preset scale or until the preset number of convolutions is reached, so as to obtain multiple second convolution results corresponding to different scales. The second sequence and the plurality of second convolution results are concatenated to obtain the second multi-scale sequence.

7. A long-term time series prediction device based on sequence decomposition and multi-scale attention mechanism, characterized in that, The device includes: Decomposition unit, used to perform sequence decomposition on the original data to obtain trend components and seasonal components; The prediction unit is used to predict the trend component to obtain a first prediction result, and to predict the seasonal component to obtain a second prediction result. The fusion unit is used to fuse the first prediction result and the second prediction result to obtain the target preset result.

8. The apparatus according to claim 7, characterized in that, The decomposition unit is used for: The original sequence is padded to obtain the padded original sequence; The trend component is obtained by processing the padded original sequence pairs using a moving average function. The seasonal component is obtained by subtracting the trend component from the original sequence.

9. The apparatus according to claim 8, characterized in that, The prediction unit is specifically used for: The trend component is predicted using linear regression to obtain the first prediction result.

10. The apparatus according to any one of claims 7 and 9, characterized in that, The prediction unit is specifically used for: The seasonal components are time-encoded to obtain a first sequence, which includes time features; and a multi-scale attention mechanism is constructed by processing a key matrix within a preset scale range according to each query matrix. Based on the first sequence, a first multi-scale sequence is constructed; the scale of each scale sequence in the first multi-scale sequence is positively correlated with the position of the scale sequence. The first multi-scale sequence is processed according to the multi-scale attention mechanism to obtain the second sequence; Based on the second sequence, a second multi-scale sequence is constructed; The second multi-scale sequence is encoded by variable dimension to obtain the third sequence; The third sequence is processed according to the multi-scale attention mechanism to obtain the fourth sequence; The fourth sequence is subjected to forward propagation and residual connection processing to obtain the second prediction result.

Images