Time sequence prediction method, computer equipment and computer readable storage medium

Abstract

The invention discloses a time sequence prediction method, computer equipment and a computer readable storage medium, and the method comprises the steps: obtaining a plurality of historical time sequences, and each historical time sequence comprises the feature data of a variable at a plurality of historical time points; a plurality of historical time sequences are input into the time sequence branches, a first result output by the time sequence branches is obtained, and the first result represents the dependency relationship between different time slices in the same historical time sequence and the dependency relationship between fine-grained sequences; the multiple historical time sequences are input into the multivariable branches, a second result output by the multivariable branches is obtained, the second result represents the inter-sequence dependency relationship of coarse granularity, and the coarse granularity is that single historical time sequences serve as a whole for interaction; and according to the first result and the second result, fusion prediction is carried out to obtain a plurality of prediction time sequences, and each prediction time sequence comprises feature data of one variable at a plurality of future time points.

Description

Technical Field

[0001] This specification relates to the field of computer technology, and in particular to a time series forecasting method, computer device, and computer-readable storage medium. Background Technology

[0002] Time series forecasting is a technique that uses analysis and modeling of time-series data to predict future values ​​or trends. It is widely used in fields such as transportation, meteorology, energy, and finance, aiming to capture characteristics such as time dependence, periodicity, and trends.

[0003] In related technologies, deep learning-based time series prediction models have developed into various technical approaches, but all of them struggle to balance the accuracy and efficiency of time series prediction. Therefore, how to balance the accuracy and efficiency of time series prediction has become an urgent technical problem to be solved. Summary of the Invention

[0004] This specification provides a time series forecasting method, computer device, and computer-readable storage medium that can balance the accuracy and efficiency of time series forecasting.

[0005] Firstly, a time series prediction method is provided, comprising: acquiring multiple historical time series, each historical time series including feature data of a variable at multiple historical time points; inputting the multiple historical time series into a time series branch to obtain a first result output by the time series branch, the first result representing the dependency relationship between different time segments within the same historical time series and the fine-grained inter-series dependency relationship, where fine-grained refers to the granularity of a single time segment; inputting the multiple historical time series into a multivariate branch to obtain a second result output by the multivariate branch, the second result representing the coarse-grained inter-series dependency relationship, where coarse-grained refers to the granularity of a single historical time series; and performing fusion prediction based on the first result and the second result to obtain multiple predicted time series, each predicted time series including feature data of a variable at multiple future time points.

[0006] In conjunction with the first aspect, in some possible implementations, the temporal branch includes: a preprocessing module, a first self-attention processing module, a second cross-attention processing module, and a second cross-attention processing module; inputting multiple historical time series into the temporal branch to obtain the first result output by the temporal branch includes: inputting multiple historical time series into the preprocessing module to preprocess each historical time series to obtain a time segment representation of each historical time series, the time segment representation including the embedding representations corresponding to multiple time segments within the historical time series; inputting the time segment representations of the historical time series into the first self-attention processing module to obtain a first context-aware embedding sequence, the first context-aware embedding sequence fusing different time slices within the historical time series. The dependencies between segments are identified; the adaptive aggregation result and the first context-aware embedding sequence are input into the second cross-attention processing module to obtain the second context-aware embedding sequence. The adaptive aggregation result is obtained by applying the inter-variable attention matrix calculated in the multivariate branch on the variable dimension and weighting the first context-aware embeddings of each historical time series. The second context-aware embedding sequence incorporates fine-grained inter-sequence dependencies. The second context-aware embedding sequence and the third context-aware embedding sequence are input into the second cross-attention processing module to obtain the first result of the time series branch output. The third context-aware embedding sequence is obtained based on the multivariate branch and incorporates coarse-grained inter-sequence dependencies.

[0007] Combining the first aspect and the above implementation methods, in some possible implementation methods, the preprocessing module includes a standardization module, a multi-scale splitting module, and a first linear projection module. Multiple historical time series are input into the preprocessing module to preprocess each historical time series, obtaining a time segment representation of each historical time series. This includes: inputting each historical time series into the standardization module, applying reversible instance normalization to each historical time series to obtain a standardized historical time series; inputting the standardized historical time series into the multi-scale splitting module, performing multi-scale segmentation on the standardized historical time series, converting the standardized historical time series into multiple time segments with location embeddings; and inputting the multiple time segments with location embeddings into the first linear projection module, mapping the multiple time segments with location embeddings to an embedding representation of a preset dimension through a first linear projection function, obtaining a time segment representation of each historical time series at each scale.

[0008] Combining the first aspect and the above implementation methods, in some possible implementation methods, the first self-attention processing module includes: a first self-attention mechanism layer and a first post-processing layer. Inputting the time segment representation of the historical time series into the first self-attention processing module to obtain the first context-aware embedding sequence includes: inputting the time segment representation of the historical time series into the first self-attention mechanism layer to obtain a first intermediate representation output by the first self-attention mechanism layer, where the first intermediate representation represents the dependency relationship between different time segments within the same historical time series; inputting the first intermediate representation and the time segment representation of the historical time series into the first post-processing layer; performing a residual connection between the first intermediate representation and the time segment representation of the historical time series through the first post-processing layer; performing layer normalization on the result of the residual connection; performing a nonlinear transformation on the normalized result through a feedforward network layer; and performing layer normalization again on the result of the nonlinear transformation to obtain the first context-aware embedding sequence.

[0009] Combining the first aspect and the above implementation methods, in some possible implementation methods, the first cross-attention processing module includes: a first cross-attention layer and a second post-processing layer. The adaptive aggregation result and the first context-aware embedding sequence are input into the first cross-attention processing module to obtain a second context-aware embedding sequence. This includes: inputting the adaptive aggregation result and the first context-aware embedding sequence into the first cross-attention layer to obtain a second intermediate representation output by the first cross-attention layer, where the second intermediate representation represents fine-grained inter-sequence dependencies; inputting the second intermediate representation and the first context-aware embedding sequence into the second post-processing layer; performing a residual connection between the second intermediate representation and the first context-aware embedding sequence through the second post-processing layer; normalizing the result of the residual connection; performing a nonlinear transformation on the normalized result through a feedforward network layer; and performing layer normalization again on the result of the nonlinear transformation to obtain the second context-aware embedding sequence.

[0010] Combining the first aspect and the above implementation methods, in some possible implementation methods, the second cross-attention processing module includes: a second cross-attention layer and a third post-processing layer. The second context-aware embedding sequence and the third context-aware embedding sequence are input into the second cross-attention processing module to obtain the first result of the temporal branch output. This includes: inputting the second context-aware embedding sequence and the third context-aware embedding sequence into the second cross-attention layer to obtain a third intermediate representation; inputting the third intermediate representation and the second context-aware embedding sequence into the third post-processing layer; performing a residual connection between the third intermediate representation and the second context-aware embedding sequence through the third post-processing layer; normalizing the result of the residual connection; performing a nonlinear transformation on the normalized result through a feedforward network layer; and performing layer normalization again on the result of the nonlinear transformation to obtain the first result.

[0011] Combining the first aspect and the above implementation methods, in some possible implementation methods, the multivariate branch includes: a second linear projection module, a second self-attention processing module, and a third cross-attention processing module. Multiple historical time series are input into the multivariate branch to obtain a second result output by the multivariate branch. This includes: inputting multiple historical time series into the second linear projection module, transforming the multiple historical time series through a second linear projection function to obtain embedded representations of the multiple historical time series; inputting the embedded representations of the multiple historical time series into the second self-attention processing module to obtain a third context-aware embedding sequence output by the second self-attention processing module, the third context-aware embedding sequence incorporating coarse-grained inter-sequence dependencies; inputting the third context-aware embedding sequence and the second context-aware embedding sequence into the third cross-attention processing module to obtain a second result, the second context-aware embedding sequence being obtained based on the temporal branch and incorporating fine-grained inter-sequence dependencies.

[0012] Combining the first aspect and the above implementation methods, in some possible implementation methods, the second self-attention processing module includes: a second self-attention layer and a fourth post-processing layer. The second self-attention processing module inputs the embedded representations of multiple historical time series to obtain the third context-aware embedded sequence output by the second self-attention processing module. This includes: inputting the embedded representations of multiple historical time series into the second self-attention layer to obtain a fourth intermediate representation, which represents coarse-grained inter-sequence dependencies; inputting the fourth intermediate representation and the embedded representations of multiple historical time series into the fourth post-processing layer; performing residual connections between the fourth intermediate representation and the embedded representations of multiple historical time series; performing layer normalization on the result of the residual connections; performing nonlinear transformation on the normalized result through a feedforward network layer; and performing layer normalization again on the result of the nonlinear transformation to obtain the third context-aware embedded sequence.

[0013] Combining the first aspect and the above implementation methods, in some possible implementation methods, the third cross-attention processing module includes: a third cross-attention layer and a fifth post-processing layer. The third context-aware embedding sequence and the second context-aware embedding sequence are input into the third cross-attention processing module to obtain the second result output by the third cross-attention processing module. This includes: inputting the third context-aware embedding sequence and the second context-aware embedding sequence into the third cross-attention layer to obtain a fifth intermediate representation; inputting the fifth intermediate representation and the third context-aware embedding sequence into the fifth post-processing layer; performing a residual connection between the fifth intermediate representation and the third context-aware embedding sequence; performing layer normalization on the result of the residual connection; performing a nonlinear transformation on the normalized result through a feedforward network layer; and performing layer normalization again on the result of the nonlinear transformation to obtain the second result.

[0014] In a second aspect, a computer device is provided, comprising one or more processors and one or more memories, wherein at least one computer program is stored in the one or more memories, and the computer program is loaded and executed by the one or more processors to implement the aforementioned time series prediction method.

[0015] Fourthly, a computer-readable storage medium is provided, wherein at least one computer program is stored in the computer-readable storage medium, the computer program being loaded and executed by a processor to implement the above-described time series prediction method.

[0016] Fifthly, a computer program product or computer program is provided, the computer program product or computer program including program code stored in a computer-readable storage medium, a processor of a computer device reading the program code from the computer-readable storage medium, the processor executing the program code, causing the computer device to perform the above-described time series prediction method.

[0017] The technical solutions provided in the embodiments of this specification propose a dual-branch collaborative prediction architecture. A time-series branch captures fine-grained intra-sequence and inter-sequence dependencies, while a multivariate branch captures coarse-grained inter-sequence dependencies, thereby improving prediction capabilities at both the fine-grained and coarse-grained levels. Through the division of labor and collaboration between these two branches, features can be extracted and fused at an appropriate granularity from both time and variable dimensions. This allows for a more comprehensive and accurate capture of key information in multivariate time series without excessively increasing computational complexity, thus improving prediction accuracy. Attached Figure Description

[0018] To more clearly illustrate the technical solutions in the embodiments of this specification, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this specification. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1 This is a schematic diagram illustrating the implementation environment of a time series prediction method provided in the embodiments of this specification; Figure 2 This is a schematic flowchart illustrating a time series prediction method provided in the embodiments of this specification; Figure 3 This is a schematic diagram illustrating three types of dependency relationships in a dual-branch architecture modeling provided in an embodiment of this application; Figure 4 This is a schematic diagram of the architecture of the prediction model provided in an embodiment of this application; Figure 5This is another schematic diagram of the prediction model provided in the embodiments of this application; Figure 6 This is a schematic diagram of the structure of a time series prediction device provided in the embodiments of this specification; Figure 7 This is a schematic diagram of the structure of a computer device provided in the embodiments of this specification. Detailed Implementation

[0020] To make the objectives, technical solutions, and advantages of this specification clearer, the embodiments of this specification will be further described in detail below with reference to the accompanying drawings.

[0021] In this manual, the terms "first," "second," etc., are used to distinguish identical or similar items with essentially the same function. It should be understood that there is no logical or temporal dependency between "first," "second," and "nth," nor are there any restrictions on quantity or execution order.

[0022] Time series forecasting is a technique that analyzes and models chronologically ordered data sequences (i.e., time series) to predict future values ​​or trends. It is widely used in transportation, meteorology, energy, finance, and other fields to capture characteristics such as time dependence, periodicity, and trends. For example, in transportation, it predicts short-term traffic flow at road sections or areas, providing a basis for traffic signal control, route guidance, and congestion warnings. In energy, it predicts electricity load or renewable energy generation capacity, supporting grid scheduling plans, electricity market transactions, and stability assurance. In meteorology, it predicts meteorological elements such as temperature and precipitation, serving agricultural production and disaster warning. In finance, it predicts the subscription and redemption volumes of wealth management products, helping financial institutions optimize resource allocation, reduce management costs, and provide scientific decision support for refined operations. Therefore, understanding how to conduct time series forecasting is particularly important.

[0023] In related technologies, time series prediction models based on deep learning have formed multiple technical routes, each with its own characteristics and limitations: Prediction methods based on convolutional neural networks (CNNs), represented by Temporal Convolutional Networks (TCNs) and TimesNet, focus on capturing local patterns, but have limited ability to model long-range dependencies.

[0024] Prediction methods based on recurrent neural networks (RNNs), represented by deep autoregressive (DeepAR) models, capture sequence dynamics through recursion, but often face problems such as unstable training and high computational complexity.

[0025] Multilayer perceptron (MLP) based prediction methods have been widely used in univariate prediction models such as DLinear and RLinear due to their simplicity and scalability, but their ability to express complex time dependencies is relatively weak.

[0026] Forecasting methods based on the Transformer architecture: The introduction of the Transformer architecture has significantly improved the capabilities of time series forecasting. LogTrans was the first to apply the Transformer to time series modeling, demonstrating its powerful potential for handling complex dependencies. Subsequent studies such as Informer and Autoformer improved computational efficiency through sampling and decomposition techniques, while PatchTST further optimized the fragment-based processing approach.

[0027] For multivariate time series (MTS) forecasting, forecasting methods can be divided into two main categories: channel-independent (CI) and channel-dependent (CD). CI (Compatibility Query) is a multivariate time series modeling approach that assumes each channel (variable) in a multivariate time series is independent, modeling and predicting each channel separately without considering potential correlations between channels. CD (Dependency Interaction) is another multivariate time series modeling approach that emphasizes the relationships between channels (variables) in a multivariate time series. It improves prediction accuracy by uncovering dependencies between channels and is suitable for scenarios with strongly correlated multivariate data. CI models each channel independently, ignoring interaction information between channels, which may result in predictions lacking a global perspective. In contrast, CD explicitly models the interactions between channels, better capturing complex dependencies in multivariate data, but its computational complexity is higher and it is prone to over-smoothing, which can negatively impact prediction accuracy.

[0028] MLP-type models (such as SOFTS, LightTS, and TSMixer) achieve efficient capture of cross-channel interactions through design, but their ability to model long-range dependencies is insufficient. Transformer-type models can model channel dependencies in multivariate time series from two perspectives. The first approach, represented by iTransformer, embeds the entire time series as a vector to model the interactions between them. While this method excels at capturing inter-channel relationships, it often ignores temporal dependencies, potentially leading to poor performance in scenarios where time patterns are crucial. The second approach, represented by Crossformer, directly captures dependencies in the time domain. However, modeling multivariate interactions at the sequence level can introduce high computational complexity and unwanted noise, thus reducing prediction accuracy.

[0029] In summary, balancing the accuracy and efficiency of time series forecasting has become a pressing technical problem that needs to be solved.

[0030] To at least address the aforementioned technical problems, embodiments of this application provide a time series forecasting method, proposing a dual-branch model architecture. This architecture captures fine-grained intra-sequence and inter-sequence dependencies through a time series branch, and captures coarse-grained inter-sequence dependencies through a multivariate branch. This improves forecasting capabilities at both the fine-grained and coarse-grained levels, balancing the accuracy and efficiency of time series forecasting.

[0031] It should be noted that all information (including but not limited to user device information, user personal information, etc.), data (including but not limited to data used for analysis, stored data, displayed data, etc.), and signals involved in this manual are authorized by the user or fully authorized by all parties, and the collection, use, and processing of related data must comply with the relevant laws, regulations, and standards of the relevant countries and regions. For example, asset information involved in this manual was obtained with full authorization.

[0032] The implementation environment of the embodiments in this specification will be described below.

[0033] Figure 1 This is a schematic diagram illustrating the implementation environment of a time series prediction method provided in the embodiments of this specification. See also... Figure 1 The implementation environment may include a prediction platform 110. The prediction platform 110 is a standalone physical server, a server cluster or distributed system consisting of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery network (CDN), and big data and artificial intelligence platforms.

[0034] The technical solutions provided in the embodiments of this specification can be applied to various scenarios involving time series forecasting, such as traffic flow forecasting, weather forecasting, energy forecasting, and financial forecasting.

[0035] Figure 2 This is a schematic flowchart illustrating a time series forecasting method provided in the embodiments of this specification. See also... Figure 2 Taking the prediction platform as the executing entity as an example, the method includes the following steps 202 to 208.

[0036] Step 202: Obtain multiple historical time series, each of which includes feature data of a variable at multiple historical time points.

[0037] Step 204: Input multiple historical time series into the time series branch to obtain the first result output by the time series branch. The first result represents the dependency relationship between different time segments within the same historical time series and the fine-grained inter-series dependency relationship. Fine-grained means using a single time segment as the granularity.

[0038] Step 206: Input multiple historical time series into the multivariate branch to obtain the second result output by the multivariate branch; wherein, the second result represents the coarse-grained inter-series dependency relationship, and coarse-grained means using a single historical time series as the granularity.

[0039] Step 208: Perform fusion prediction based on the first result and the second result to obtain multiple predicted time series. Each predicted time series includes feature data of a variable at multiple future time points.

[0040] exist Figure 2 The illustrated embodiment proposes a dual-branch collaborative prediction architecture. A time-series branch captures fine-grained intra-sequence and inter-sequence dependencies, while a multivariate branch captures coarse-grained inter-sequence dependencies, thus improving prediction capabilities at both fine-grained and coarse-grained levels. Through the division of labor and collaboration between these two branches, features can be extracted and fused at an appropriate granularity from both time and variable dimensions. This allows for a more comprehensive and accurate capture of key information in multivariate time series without excessively increasing computational complexity, thereby improving prediction accuracy and effectively addressing the technical problems of existing CI methods ignoring correlations and CD methods being inefficient and prone to oversmoothing.

[0041] The following is about Figure 2 The specific implementation methods of each step in the illustrated embodiment are explained below: In step 202, the prediction platform acquires multiple historical time series. Each historical time series includes feature data of a variable at multiple historical time points. Each historical time series represents a single variable, and the multiple historical time series represent various variables. The feature data can be observed or calculated values ​​of the variable. Each historical time series represents the historical variation pattern of a variable, and in a specific implementation, each historical time series can be described using vectors. The multiple historical time series collectively represent various influencing factors related to the prediction target.

[0042] The specific type of variable depends on the prediction scenario. The following section introduces variable types in different prediction scenarios: In traffic flow prediction scenarios, multiple variables are used to describe the traffic conditions and influencing factors of a target road segment or area. These variables include: traffic volume, speed, road conditions, weather conditions, and time attributes. Multiple historical time series are included: historical time series 1 for traffic volume, historical time series 2 for speed, historical time series 3 for road conditions, historical time series 4 for weather conditions, and historical time series 5 for time attributes. Historical time series 1 includes observations of traffic volume at multiple historical time points for the target road segment. Historical time series 2 includes observations of speed at multiple historical time points for the target road segment. Historical time series 3 includes observations of road conditions (e.g., smooth flow, slow traffic, congestion) at multiple historical time points for the target road segment. Historical time series 4 includes observations of weather conditions at multiple historical time points for the target road segment. Historical time series 5 includes whether each historical time point was a weekday or holiday.

[0043] In meteorological element (such as temperature) forecasting scenarios, multiple variables are used to reflect the atmospheric physical state that influences changes in the target meteorological element. These variables include temperature, humidity, sea-level pressure, and wind speed. Multiple historical time series are included: historical time series 1 for temperature, historical time series 2 for humidity, historical time series 3 for sea-level pressure, and historical time series 4 for wind speed. Historical time series 1 includes observed temperature values ​​at multiple historical time points. Historical time series 2 includes observed humidity values ​​at multiple historical time points. Historical time series 3 includes observed sea-level pressure values ​​at multiple historical time points (e.g., unobstructed, slow-moving, congested, etc.). Historical time series 4 includes observed wind speed values ​​at multiple historical time points.

[0044] In the scenario of electricity load forecasting, multiple variables are used to describe the electricity consumption characteristics that influence changes in electricity demand. These variables include: power load, time attributes, and electricity consumption. Multiple historical time series are included: historical time series 1 corresponding to power load, historical time series 2 corresponding to time attributes, and historical time series 3 corresponding to electricity consumption. Historical time series 1 includes observed power load values ​​of the regional power grid at multiple historical time points. Historical time series 2 includes whether each historical time point was a weekday or holiday. Historical time series 3 includes electricity consumption at multiple historical time points.

[0045] In the scenario of predicting the subscription and redemption volume of wealth management products, multiple variables include: product yield, product limit, and time attributes. Multiple historical time series are included: historical time series 1 corresponding to the product yield, historical time series 2 corresponding to the product limit, and historical time series 3 corresponding to the time attributes. Historical time series 1 includes characteristic data of the product yield at multiple historical points in time (such as the value of the product yield). Historical time series 2 includes characteristic data of the product limit at multiple historical points in time (such as the value of the product limit). Historical time series 3 includes whether each historical point in time was a weekday / holiday.

[0046] By acquiring the aforementioned multivariate historical time series related to the prediction scenario, the prediction platform provides a complete data foundation for subsequently building a deep learning model that can integrate time-series dependencies and multivariate interactions.

[0047] In steps 204 and 206, after the prediction platform acquires multiple historical time series, it inputs them in parallel into the two core branches of the prediction model: the temporal branch and the multivariate branch, to obtain the first result output by the temporal branch and the second result output by the multivariate branch, respectively.

[0048] The first result can simultaneously characterize two types of dependencies: (1) the dependencies between different time segments within the same historical time series, i.e. the time evolution pattern within the series; (2) fine-grained inter-series (i.e., cross-variable) dependencies. Fine-grained means taking a single time segment in each series as the basic interaction unit, treating a single time segment as a whole, and capturing the local correlation of variables on the time segment.

[0049] The second result mainly characterizes coarse-grained inter-series (cross-variable) dependencies. Coarse-grained means taking each complete historical time series as a basic interaction unit and treating each complete historical time series as a whole (i.e. a variable representation vector), directly modeling the dependencies between variables.

[0050] The time-series branch and the multivariate branch together constitute the dual-branch architecture of the prediction model deployed by the prediction platform. The prediction model in this embodiment can be a Transformer model based on the dual-branch architecture. The model is trained using a preset training set, which contains several sample time series and their labeled sample labels. The sample labels are the prediction target sequences (representing the true values ​​to be predicted) for a future period corresponding to the sample time series.

[0051] The dual-branch architecture provided in this application can model three types of dependencies, including: (1) temporal dependencies within a single sequence (i.e., time patterns within the sequence), which are the dynamic change patterns of a single variable in the time dimension; (2) coarse-grained inter-channel dependencies, which abstract the entire time series corresponding to each variable into a whole representation, thereby mining the global correlation between variables; and (3) fine-grained inter-channel temporal dependencies, which focus on the local interaction between time steps and sequences, paying attention to the interaction of different variables at the specific time segment level, and are a more refined cross-variable local dependency. Temporal dependencies within a single sequence can be understood as the dependencies between different time segments within the same historical time series mentioned above. Coarse-grained inter-channel dependencies can be understood as the coarse-grained inter-sequence dependencies mentioned above. Fine-grained inter-channel temporal dependencies can be understood as the fine-grained inter-sequence dependencies mentioned above.

[0052] For example, targeted attention mechanisms can be designed in both branches to effectively capture the three types of nominal dependencies mentioned above. For instance, the time-series branch employs a multi-resolution, segment-level self-attention mechanism, first dividing each historical time-series sequence into time segments (patches) of different scales, and then performing self-attention computation at the segment level. Through multi-resolution design, the predictive model can simultaneously capture short-term local fluctuations and long-term cyclical trends, thereby efficiently modeling time-series dependencies within a single sequence and naturally establishing fine-grained cross-variable associations at the segment level. The multivariate branch can employ a variable-level attention mechanism, directly focusing on the global correlations between variables, thereby modeling coarse-grained inter-channel dependencies.

[0053] By parallel processing of time-series branches and multivariate branches and subsequent feature fusion, the prediction model of this application embodiment can simultaneously take into account the multi-scale evolution law of the time dimension and the global and local interaction relationship of the variable dimension. It overcomes the shortcomings of channel-independent methods that ignore variable correlations and channel-dependent methods that are computationally inefficient or susceptible to time step noise, thereby improving the accuracy and robustness of multivariate time series prediction.

[0054] Figure 3 This is a schematic diagram illustrating three types of dependency relationships in a dual-branch architecture model provided in an embodiment of this application.

[0055] For example, such as Figure 3 The diagram schematically illustrates time series 301, 302, and 303. Time segment 3011 is derived from time series 301; time segments 3021, 3022, 3023, and 3024 are derived from time series 302; and time segment 3031 is derived from time series 303. Temporal dependencies within a single series can include the temporal dependencies between time segments 3021, 3022, and 3023 in time series 302. Coarse-grained inter-channel dependencies can include the dependencies between time series 301, 302, and 303. Fine-grained inter-channel temporal dependencies can include the temporal dependencies between time segments 3011 in time series 301, time segment 3024 in time series 302, and time segment 3031 in time series 303.

[0056] Figure 3 Time series 301, time series 302, and time series 303 can be understood as three historical time series corresponding to the three types of variables mentioned above.

[0057] Figure 4 This is a schematic diagram of the architecture of the prediction model provided in the embodiments of this application.

[0058] For example, such as Figure 4 The prediction model includes: a preprocessing module 40, a second linear projection module 44, L stacked layers, and a fusion and prediction module 47. Each of the L stacked layers receives the output of the previous layer as input and passes its output to the next layer. The output of the last layer serves as the input to the fusion and prediction module 47. Each layer includes: a first self-attention processing module 41, a first cross-attention processing module 42, a second cross-attention processing module 43, a second self-attention processing module 45, and a third cross-attention processing module 46.

[0059] In some embodiments, such as Figure 4 As shown, the temporal branch includes: a preprocessing module 40, a first self-attention processing module 41, a first cross-attention processing module 42, and a second cross-attention processing module 43. Step 204 above includes the following steps 2041 to 2044: Step 2041: Input multiple historical time series into the preprocessing module 40 to preprocess each historical time series to obtain a time segment representation of each historical time series. The time segment representation includes the embedded representations corresponding to multiple time segments within the historical time series.

[0060] Specifically, for each input univariate historical time series, it is divided into several consecutive and potentially overlapping fixed-length subsequences along the time dimension, called time segments. For each of these time segments, a learnable linear projection layer or a one-dimensional convolutional layer is used to map it into a high-dimensional vector representation, called segment embedding. Arranging all time segments of a historical time series in their original chronological order, the embedding representations corresponding to multiple time segments constitute the time segment representation of that historical time series.

[0061] Figure 5 This is another schematic diagram of the prediction model provided in the embodiments of this application.

[0062] In some embodiments, such as Figure 5 As shown, the preprocessing module includes a normalization processing module 401, a multi-scale splitting module 402, and a first linear projection module 403. Step 2041 includes the following steps S11 to S13: S11: Input each historical time series into the standardization processing module 401, apply reversible instance normalization to each historical time series, and obtain a standardized historical time series.

[0063] Specifically, the standardization module 401 applies Reversible Instance Normalization (RevIN) to each historical time series. It calculates the mean and standard deviation of the series within the historical observation window, then subtracts the mean from the observed value at each time point in the series and divides it by the standard deviation to obtain a normalized series with zero mean and unit variance—that is, the standardized historical time series. This operation aims to eliminate scale differences and distribution drift that may exist between different time series, and between the training and testing phases of the same series, thereby improving the stability and generalization ability of the model training.

[0064] During the training phase, reversible instance normalization is performed on the input sequence to stabilize the training process. Simultaneously, normalization parameters are recorded or learned to mitigate distribution drift between training and test data. In the prediction phase, the same normalization operation is performed on the input sequence, and the mean and standard deviation used for normalization are recorded. After the prediction model completes its predictions, these parameters are used to perform inverse normalization (i.e., multiplying by the standard deviation and adding the mean) to restore the prediction model's output to its original physical scale, thus ensuring the practical interpretability and direct usability of the prediction results.

[0065] S12: Input the standardized historical time series into the multi-scale segmentation module 402, perform multi-scale segmentation on the standardized historical time series, and convert the standardized historical time series into multiple time segments with location embedding.

[0066] Specifically, the multi-scale segmentation module 402 is used to perform multi-scale segmentation on the standardized historical time series to transform the standardized historical time series into multiple time segments with location embeddings. In this embodiment, M scales (i.e., M resolutions) can be used for processing. The segment size determines the ability to capture time patterns. Shorter time segments can capture high-frequency local patterns, while longer time segments are used to model long-term trends.

[0067] For example, for a resolution of m, a historical time series of length T is divided into segments of length m. Generate continuous time segments There are 10 segments, among which This represents the sliding step size. For each standardized historical time series of a single variable, it is divided in parallel along the time axis using multiple sliding step sizes of different scales (i.e., different lengths), generating multiple sets of time segments, each corresponding to a specific scale. For resolution m, the i-th historical time series... The time segments are divided into a set of time segments with location embeddings using the following formula (1). : Formula (1) in, It can also be understood as the historical time series vector of the i-th variable, including the observation values ​​of the i-th variable at multiple historical time points.

[0068] S13: Input multiple time segments with location embedding into the first linear projection module 403, and map the multiple time segments with location embedding into an embedding representation of a preset dimension through the first linear projection function to obtain the time segment representation of each historical time series at each scale.

[0069] Specifically, for each obtained time segment, the scalar sequence within each time segment is mapped to a high-dimensional segment embedding vector using the first linear projection function. Learnable positional embeddings are then added to each segment embedding vector. Since the temporal order information of the original sequence (i.e., the historical time series) may be weakened after segmentation and recombination, the introduction of positional embeddings aims to explicitly inject information about the position (or order) of each segment in the original sequence, ensuring that the model can understand and utilize the temporal structure of the sequence.

[0070] For example, these time segments Embedded through linear transformation In the middle. Among them, Let represent the first trainable linear projection function, and D be the embedding dimension. This represents the i-th historical time series at scale m. The embedded representation (i.e., the time segment representation of the i-th variable) is obtained by the following formula (2): Formula (2) Assuming there are N historical time series, the time segments of these N historical time series at scale m are denoted as follows: , For the first layer in the L layers of input stack ( =0 indicates the time segment representation of the first layer. That is, the time segment representation of the input to the first self-attention processing module 41 of the first layer is... At scale m, the second layer ( =1 indicates the input of the first self-attention processing module 41 in the second layer. This is the output of the second cross-attention processing module 43 in layer 1, and so on. This is the output of the second cross-attention processing module 43 in the last layer at scale m.

[0071] In the above embodiments, through the processing of the standardization module, the multi-scale splitting module, and the first linear projection module, the original historical time series is reorganized into a fragment-level representation containing local semantics. This not only reduces the sequence length that the subsequent self-attention mechanism needs to process and improves computational efficiency, but more importantly, it lays the foundation for the subsequent multi-resolution fragment-level self-attention mechanism of the temporal branch, enabling it to more effectively capture dependency patterns at different time scales within the sequence and fine-grained cross-variable temporal dependencies.

[0072] Step 2042: Input the time segment representation of the historical time series into the first self-attention processing module 41 to obtain the first context-aware embedding sequence, which integrates the dependencies between different time segments in the historical time series.

[0073] The first self-attention processing module 41 is the self-attention processing module for the temporal branch, and therefore can also be called the temporal self-attention processing module. The time segment representation sequence of each historical time series is input to the first self-attention processing module 41 of the temporal branch (i.e., the temporal self-attention processing module) for processing based on a multi-resolution segment-level self-attention mechanism, resulting in a first context-aware embedding sequence corresponding to each historical time series. This first context-aware embedding sequence integrates the dependencies within its corresponding historical time series and across different time segments.

[0074] In some embodiments, such as Figure 5As shown, the first self-attention processing module 41 includes a first self-attention layer 411 and a first post-processing layer 412. The above step 2042 includes: inputting the time segment representation of the historical time series into the first self-attention layer 411 to obtain a first intermediate representation output by the first self-attention layer 411, wherein the first intermediate represents the dependency relationship between different time segments within the same historical time series; inputting the first intermediate representation and the time segment representation of the historical time series into the first post-processing layer 412; performing residual connection between the first intermediate representation and the time segment representation of the historical time series; performing layer normalization on the result of the residual connection; performing nonlinear transformation on the normalized result through a feedforward network layer; and performing layer normalization again on the result of the nonlinear transformation to obtain the first context-aware embedding sequence.

[0075] The first self-attention layer 411, also known as the temporal self-attention layer, receives the time segment representation as input. This layer is computed based on a multi-resolution segment-level self-attention mechanism (SA), and its output is a first intermediate representation. This first intermediate representation dynamically aggregates dependency information within the same historical time series across different time segments and scales. Then, the first intermediate representation and the original time segment representation of the historical time series are input together into the first post-processing layer 412. The processing steps of the first post-processing layer 412 are as follows: First, a residual connection is formed between the first intermediate representation and the time segment representation to alleviate the gradient vanishing problem in deep networks and preserve the original information. Then, layer normalization (LN) is applied to the result of the residual connection. The normalized result is then input into a feedforward network (FFN) layer for nonlinear transformation and enhancement of the features. Finally, layer normalization is applied again to the result of the nonlinear transformation. The final output of this normalization is the first context-aware embedding sequence.

[0076] For example, in the first At layer m and scale m, the output of the first self-attention processing module can be expressed by the following formula (3): Formula (3) in, Indicates the stacked first There are L layers in total. The output of the layer is also +1 layer input. Indicates the first At a layer and scale m, the time segments of N historical time series are represented, which also represent the input of the first self-attention processing module. This refers to the first intermediate representation mentioned above. Indicates the first The output of the first self-attention processing module at layer m and scale m.

[0077] Step 2043: Input the adaptive aggregation result and the first context-aware embedding sequence into the first cross-attention processing module 42 to obtain the second context-aware embedding sequence.

[0078] The adaptive aggregation result is obtained by applying the inter-variable attention matrix calculated in the multivariate branch to the first context-aware embeddings of each historical time series in a weighted aggregation along the variable dimension. The second context-aware embedding sequence incorporates fine-grained inter-sequence dependencies, and the inter-variable attention matrix reflects the relative importance of each variable in the current context. For each target variable, the context-aware embeddings of all variables at the same scale are dynamically aggregated using the inter-variable attention matrix, i.e., the first context-aware embeddings of all historical time series. This process can be performed independently at each time scale, ultimately yielding the adaptively aggregated multivariate representation, i.e., the adaptive aggregation result.

[0079] Furthermore, to effectively manage channel interactions at the fine-grained time-step level, this embodiment proposes an adaptive multi-channel data aggregation method. This method merges multi-channel data before interaction, where multi-channel means multi-variable. For a dataset containing C channels and a sequence length of T, directly calculating the interaction complexity between all time-step pairs is O(C0). 2 T 2 This not only incurs significant computational overhead but also easily leads to overfitting.

[0080] In this embodiment, before performing fine-grained time-step interactions, adaptive aggregation is first performed on the multi-channel data. The variable corresponding to each historical time series is used as the target variable, and the aggregation is weighted according to the correlation strength between each variable and the target variable. Variables with high correlation are given higher weights in the aggregation, thereby ensuring that key information for prediction is preserved and strengthened. By aggregating multi-channel data into a single channel for each variable, the interaction complexity between the target sequence and the aggregated sequence is reduced to O(T). 2 The total complexity is O(CT). 2 This is comparable to PatchTST, which assumes no channel interaction. Furthermore, to preserve predictive information, channels are weighted and aggregated based on their relevance to each sequence, with strongly correlated channels assigned higher weights. Conveniently, the channel-level attention matrix (i.e., the inter-variable attention matrix) in the multivariate branch provides relevant information, enabling aggregation guided by the inter-variable attention matrix.

[0081] For example, the above adaptive multi-channel data aggregation process is performed at each layer and scale of the time series branch. Its core is to use the inter-variable attention matrix provided by the multivariable branch to fuse and compress the multi-channel data of the time series branch.

[0082] With the variable identifier i omitted, the timing branch is at the . The adaptive aggregation formula for layer and scale m can be found in the following formula (4): Formula (4) in, Represents the attention matrix between variables. Indicates the first The first context-aware embedding of N historical time series at layer m and scale m. Indicates the first Adaptive aggregation results at layer and scale m.

[0083] In some embodiments, a more refined version of formula (4) above can be the following formula (5): Formula (5) in, Indicates the first At layer m and scale m, a time-series feature sequence incorporating cross-variable information is generated for the i-th variable. Its shape is (Nm, D), representing Nm time segments, each a D-dimensional vector. The i-th variable corresponds to the i-th historical time series. Based on this... It can also be understood as the adaptive aggregation result corresponding to the i-th historical time series, or as the adaptive aggregation result corresponding to the i-th variable. Indicates the first The layer represents the attention weight (or correlation strength) of variable j to variable i, calculated from the multivariate branch. A higher weight indicates that, at the global level, information from variable j is more important for understanding and predicting variable i. Indicates the first At layer m and scale m, the deep temporal feature sequence obtained after the j-th variable undergoes its own temporal self-attention mechanism contains the complex temporal dependency pattern of variable j itself.

[0084] At each scale of each layer, the adaptive aggregation result and the first context-aware embedding sequence are input into the first cross-attention processing module 42 to obtain the second context-aware embedding sequence corresponding to N historical time series. The second context-aware embedding sequence incorporates fine-grained inter-sequence dependencies.

[0085] In some embodiments, such as Figure 5As shown, the first cross-attention processing module includes a first cross-attention layer 421 and a second post-processing layer 422. The above step 2043 includes: inputting the adaptive aggregation result and the first context-aware embedding sequence into the first cross-attention layer 421 to obtain a second intermediate representation output by the first cross-attention layer, wherein the second intermediate representation represents fine-grained inter-sequence dependencies; inputting the second intermediate representation and the first context-aware embedding sequence into the second post-processing layer 422; performing residual connection between the second intermediate representation and the first context-aware embedding sequence through the second post-processing layer 422; normalizing the result of the residual connection; performing nonlinear transformation on the normalized result through a feedforward network layer; and performing layer normalization again on the result of the nonlinear transformation to obtain the second context-aware embedding sequence.

[0086] Specifically, in the At layer scale m, the first context-aware embedding sequence and the adaptive aggregation result are input together into the first cross-attention layer 421 for cross-attention calculation. The first context-aware embedding sequence is used as the query, and the adaptive aggregation result is used as both the key and value. The calculation is performed through the cross-attention (CA) mechanism to obtain the second intermediate representation. The first cross-attention layer is related to the temporal branch and adaptive aggregation, and can also be called the Adaptive Aggregation Temporal Attention layer.

[0087] Then, the aforementioned second intermediate representation and the first context-aware embedding sequence are input into the second post-processing layer 422. The processing steps of the second post-processing layer 422 are as follows: First, the first intermediate representation and the first context-aware embedding sequence are residually connected. Then, layer normalization (LN) is applied to the result of the residual connection, and the result of layer normalization is input into a feedforward network layer (FFN) to perform nonlinear transformation and enhancement on the features. Finally, layer normalization is performed again on the result of the nonlinear transformation, and the final output of this normalization is the second context-aware embedding sequence.

[0088] For example, at scale m, the output of the first cross-attention processing module can be represented by the following formula (6): Formula (6) in, Indicates the first At a layer and scale m, the first context-aware embedding sequence of N historical time series is the output of the first self-attention processing module. Indicates the first The adaptive aggregation results corresponding to N historical time series at a layer and scale m. Indicates the first At a layer and scale m, there are N historical time series corresponding to the second context-aware embedding sequence. Since the second context-aware embedding sequence incorporates fine-grained inter-sequence dependencies, the second cross-attention processing module that generates the second context-aware embedding sequence can also be called a cross-sequence fine-grained cross-attention processing module.

[0089] Step 2044: Input the second context-aware embedding sequence and the third context-aware embedding sequence into the second cross-attention processing module 43 to obtain the first result of the temporal branch output. The third context-aware embedding sequence is obtained based on multivariate branching and incorporates coarse-grained inter-sequence dependencies.

[0090] The second context-aware embedding sequence comes from the temporal branch, and the third context-aware embedding sequence comes from the multivariate branch. Based on this, the second cross-attention processing module 43 can also be called a variable-to-temporal cross-attention module. This temporal cross-attention module uses variable representations as queries and uses the time segment representations of each historical time series as keys and values. This configuration allows each time step in the sequence to obtain a global view of the entire time series. The third context-aware embedding sequence can be the result output by the second self-attention processing module in the multivariate branch.

[0091] In some embodiments, such as Figure 5 As shown, the second cross-attention processing module includes a second cross-attention layer 431 and a third post-processing layer 432. The above step 2044 includes: inputting the second context-aware embedding sequence and the third context-aware embedding sequence into the second cross-attention layer 431 to obtain a third intermediate representation; inputting the third intermediate representation and the second context-aware embedding sequence into the third post-processing layer 432; performing a residual connection between the third intermediate representation and the second context-aware embedding sequence through the third post-processing layer 432; normalizing the result of the residual connection; performing a nonlinear transformation on the normalized result through a feedforward network layer; and performing layer normalization again on the result of the nonlinear transformation to obtain a first result.

[0092] Specifically, the second cross-attention layer can be called the variable-to-temporal cross-attention layer. In the... At layer scale m, the second and third context-aware embedding sequences are input together into the second cross-attention layer 431 for cross-attention calculation. The third context-aware embedding sequence is used as the query, and the second context-aware embedding sequence is used as both the key and value. The calculation is performed using a cross-attention (CA) mechanism to obtain the third intermediate representation. Then, the third intermediate representation and the second context-aware embedding sequence are input together into the third post-processing layer 432. The processing steps of the third post-processing layer 432 are as follows: First, the third intermediate representation and the second context-aware embedding sequence are joined using a residual connection. Then, layer normalization (LN) is applied to the result of the residual connection. The normalized result is then input into a feedforward network layer (FFN) for nonlinear transformation and enhancement of the features. Finally, layer normalization is applied again to the result of the nonlinear transformation to obtain the final normalized output. If the final normalized output is the output of the last layer and the Lth layer of the third post-processing layer 432, then this output is the first result.

[0093] For example, when the scale parameter m is omitted, the output of the second cross-attention processing module can be represented by the following formula (7): Formula (7) in, Indicates the first The output of the second cross-attention processing module 43 of the layer, Indicates the first The output of the first cross-attention processing module 42 of the layer, Indicates the first The output of the second self-attention processing module 45 of the layer. When hour, This is the first result of the timing branch output.

[0094] It should be noted that the multi-scale parameter is omitted in the above formula (7) for the sake of simplicity. Different heads of the cross-attention calculate the interaction with the time series representation at different scales. By integrating these two complementary directions, this bidirectional method bridges the gap between temporal dependency and variable dependency modeling, while ensuring a comprehensive and unified representation.

[0095] In step 206, multiple historical time series are input into the multivariate branch to obtain the second result of the multivariate branch output. The second result represents the coarse-grained inter-series dependencies. Coarse-grained means treating each historical time series as a whole.

[0096] In some embodiments, such as Figure 4As shown, the multivariate branch includes: a second linear projection module 44, a second self-attention processing module 45, and a third cross-attention processing module 46. Step 206 above includes the following steps 2061 to 2063: Step 2061: Input multiple historical time series into the second linear projection module, and transform the multiple historical time series through the second linear projection function to obtain the embedded representation of multiple historical time series.

[0097] The embedding representation of multiple historical time series can be understood as: the entire historical time series corresponding to multiple variables are aggregated into a feature vector.

[0098] Specifically, multivariate branching is used to model the dependencies between variables. Each historical time series... Treated as a whole, through a trainable second linear projection function (T is the sequence length, D is the embedding dimension) Embedded into In the middle, see the following formula (8): Formula (8) Suppose there are N historical time series, and the embedding representation of the N historical time series is denoted as . The embedding is represented as the first layer in an L-layer stack of inputs ( 0 represents the embedding representation of the first layer. That is, the embedding representation of the second self-attention processing module 45 input to the first layer is: The second floor ( 1 represents the input of the second self-attention processing module 45 (in the second layer). This is the output of the third cross-attention processing module in layer 1, and so on. This is the output of the third cross-attention processing module in the last layer.

[0099] Step 2062: Input the embedded representations of multiple historical time series into the second self-attention processing module 45 to obtain the third context-aware embedding sequence output by the second self-attention processing module 45. The third context-aware embedding sequence incorporates coarse-grained inter-sequence dependencies.

[0100] The second self-attention processing module 45 is a self-attention processing module for multivariate branches, and therefore can also be called a multivariate self-attention processing module. The embedded representations of multiple historical time series are input into the second self-attention processing module 45 for processing based on the self-attention mechanism to obtain a third context-aware embedding sequence. This third context-aware embedding sequence incorporates coarse-grained inter-sequence dependencies, that is, the mutual dependencies between variables.

[0101] In some embodiments, such as Figure 5 As shown, the second self-attention processing module includes a second self-attention layer 451 and a fourth post-processing layer 452. Step 2062 is implemented by: inputting the embedded representations of multiple historical time series into the second self-attention layer 451 to obtain a fourth intermediate representation, which represents coarse-grained inter-sequence dependencies. The fourth intermediate representation and the embedded representations of multiple historical time series are input into the fourth post-processing layer 452. A residual connection is performed between the fourth intermediate representation and the embedded representations of multiple historical time series. The result of the residual connection is layer-normalized. The normalized result is nonlinearly transformed through a feedforward network layer. The result of the nonlinear transformation is layer-normalized again to obtain the third context-aware embedded sequence.

[0102] The second self-attention layer 451, also known as the multivariate self-attention layer, receives the embedded representations of multiple historical time series (i.e., multivariate embedded representations) as input. This layer performs computation based on the self-attention mechanism between variables, and its output is the fourth intermediate representation. This fourth intermediate representation dynamically incorporates the dependency information between different variables. Then, the above fourth intermediate representation, along with the original embedded representation, is input to the fourth post-processing layer 452. The processing steps of the fourth post-processing layer 452 are as follows: First, a residual connection is made between the fourth intermediate representation and the embedded representation. Layer normalization (LN) is applied to the result of the residual connection. The normalized result is then input into a feedforward network layer (FFN) for nonlinear transformation and enhancement of the features. Finally, layer normalization (LN) is applied again to the result of the nonlinear transformation. The final output of this normalization is the third context-aware embedding sequence.

[0103] For example, in the first The output of the second self-attention processing module can be represented by the following formula (9): Formula (9) in, Indicates the first The input to the second self-attention processing module of the layer, Indicates the first The output of the second self-attention processing module of the layer.

[0104] Step 2063: Input the third context-aware embedding sequence and the second context-aware embedding sequence into the third cross-attention processing module 46 to obtain the second result. The second context-aware embedding sequence is obtained based on the above-mentioned temporal branch.

[0105] It is understandable that the third context-aware embedding sequence comes from the multivariate branch, and the second context-aware embedding sequence comes from the temporal branch. Based on this, the third cross-attention processing module can also be called the temporal-to-variate cross attention module. This temporal-to-variate cross attention module uses the time segment representation of each historical time series as a query and the variable representation as a key and value, so that the variable representation can effectively capture the relevant information of key time steps in the sequence.

[0106] In some embodiments, such as Figure 5 As shown, the third cross-attention processing module includes a third cross-attention layer 461 and a fifth post-processing layer 462. The above step 2063 includes: inputting the third context-aware embedding sequence and the second context-aware embedding sequence into the third cross-attention layer 461 to obtain the fifth intermediate representation; inputting the fifth intermediate representation and the third context-aware embedding sequence into the fifth post-processing layer 462; performing residual connection on the fifth intermediate representation and the third context-aware embedding sequence; performing layer normalization on the result of the residual connection; performing nonlinear transformation on the normalized result through a feedforward network layer; and performing layer normalization again on the result of the nonlinear transformation to obtain the second result.

[0107] Specifically, the third cross-attention layer can be called the Temporal-to-Variate Cross Attention layer. In the... Below this layer, the third context-aware embedding sequence and the second context-aware embedding sequence are input together into the third cross-attention layer 461 for cross-attention calculation. The second context-aware embedding sequence is used as the query, and the third context-aware embedding sequence is used as both the key and value. The calculation is performed using a cross-attention (CA) mechanism to obtain the fifth intermediate representation. Then, the fifth intermediate representation and the third context-aware embedding sequence are input together into the fifth post-processing layer 462. The processing steps of the fifth post-processing layer 462 are as follows: First, the fifth intermediate representation and the third context-aware embedding sequence are joined using a residual connection. Then, layer normalization (LN) is applied to the result of the residual connection. The normalized result is then input into a feedforward network layer (FFN) for nonlinear transformation and enhancement of the features. Finally, layer normalization is applied again to the result of the nonlinear transformation to obtain the final normalized output. If the final normalized output is the output of the last layer and the Lth layer of the fifth post-processing layer 462, then this output is the second result.

[0108] For example, the output of the third cross-attention processing module can be represented by the following formula (10): Formula (10) in, Indicates the first The output of the third cross-attention processing module of the layer, Indicates the first The output of the first cross-attention processing module 42 of the layer, Indicates the first The output of the second self-attention processing module 45 of the layer. When hour, This is the second result of the multivariate branch output.

[0109] In step 208, see Figure 4 or Figure 5 The first and second results are input into the fusion and prediction module 47 of the prediction model. The fusion and prediction module 47 combines the first and second results to predict multiple predicted time series. Each predicted time series includes feature data of a variable at multiple future time points. The multiple predicted time series are generated by applying linear projection to the concatenation of the first and second results.

[0110] For example, the prediction results output by the fusion and prediction module It can be expressed using the following formula (11): Formula (11) If there are a total of M scales, This represents the output of the second cross-attention processing module in the Lth layer at the first scale. This represents the output of the second cross-attention processing module in the Lth layer at the Mth scale. This represents the output of the third cross-attention processing module in layer L.

[0111] For example, when training a prediction model, the loss function can be defined as the true value. Compared with the predicted value The mean squared error between them. The loss function can be expressed as the following formula (12).

[0112] Formula (12) For example, as shown above, in a traffic flow prediction scenario, multiple variables include: traffic flow, traffic speed, road conditions, weather conditions, and time attributes of the target road segment. Multiple prediction time series include: prediction time series 1 for traffic flow, prediction time series 2 for traffic speed, prediction time series 3 for road conditions, prediction time series 4 for weather conditions, and prediction time series 5 for time attributes. Based on these five prediction time series, traffic flow within future time periods corresponding to multiple future time points can be predicted.

[0113] For example, as shown above, in a meteorological element (such as temperature) prediction scenario, multiple variables include: temperature, humidity, sea level pressure, wind speed, etc. Multiple prediction time series are included: prediction time series 1 for temperature, prediction time series 2 for humidity, prediction time series 3 for sea level pressure, and prediction time series 4 for wind speed. Based on these four prediction time series, the temperature conditions for future time periods corresponding to multiple future time points can be predicted.

[0114] For example, as shown above, in a power load forecasting scenario, multiple variables include: power load, time attributes, and electricity consumption. Multiple forecast time series include: forecast time series 1 corresponding to power load, forecast time series 2 corresponding to time attributes, and forecast time series 3 corresponding to electricity consumption. Based on these three forecast time series, the power load for future time periods corresponding to multiple future time points can be predicted.

[0115] In summary, the dual-branch architecture design and bidirectional cross-attention mechanism in this application effectively overcome the main shortcomings of existing multivariate time series prediction methods and achieve significant technical improvements in several aspects: Firstly, this application addresses the issue of channel-independent (CI) methods neglecting cross-channel interactions: traditional CI methods model each channel independently, ignoring inter-channel dependencies. This application's embodiments explicitly model cross-channel dependencies through a multivariate branching channel-level self-attention mechanism, while simultaneously combining a bidirectional cross-attention mechanism to integrate multivariate interaction information with temporal patterns, thereby comprehensively capturing global cross-channel dependencies.

[0116] Secondly, this paper addresses the high complexity and oversmoothing issues of channel dependency (CD) methods: While traditional CD methods can model inter-channel dependencies, they often suffer from high computational complexity and oversmoothing. This application's embodiments employ an adaptive multi-channel aggregation strategy, guided by a channel-level attention matrix, to aggregate multi-channel data into a single hybrid channel. This significantly reduces model complexity while preserving key information and avoiding the weakening of individual channel features due to oversmoothing.

[0117] Thirdly, it enhances the collaborative modeling capability of temporal patterns and multivariate interactions: traditional methods often struggle to simultaneously handle fine-grained modeling of temporal dependencies and cross-channel interactions. This application's embodiments capture temporal patterns through multi-resolution fragment-level attention in temporal branches, and combine this with a bidirectional cross-attention mechanism to achieve mutual enhancement of variables and temporal features. Variable-to-temporal attention (the second cross-attention processing module) transmits multivariate interactions to the temporal representation, while temporal-to-variable attention (the third cross-attention processing module) integrates the temporal pattern into the variable representation, achieving cross-branch mutual information enhancement, thus achieving a better balance in dynamic interactions across different time scales and channels.

[0118] Fourthly, this application improves the balance between computational efficiency and model performance: through a dual-branch architecture and an adaptive aggregation strategy, the embodiments of this application reduce computational complexity while ensuring efficient modeling of long-range dependencies, global relationships, and local interactions. This design not only improves prediction accuracy but also enhances the model's scalability, making it more suitable for processing high-dimensional, multivariate time series data.

[0119] All the above-described optional implementation methods can be combined in any way to form optional embodiments of this specification, and will not be described in detail here.

[0120] Figure 6 This is a schematic diagram of the structure of a time series prediction device provided in the embodiments of this specification. See also... Figure 6 The time series prediction device 600 includes: The acquisition module 601 is used to acquire multiple historical time series, each of which includes feature data of a variable at multiple historical time points.

[0121] The first processing module 602 is used to input multiple historical time series into the time series branch and obtain the first result output by the time series branch; wherein, the first result represents the dependency relationship between different time segments within the same historical time series and the fine-grained inter-series dependency relationship, and fine-grained means taking a single time segment as the granularity.

[0122] The second processing module 603 is used to input multiple historical time series into the multivariate branch and obtain the second result output by the multivariate branch; wherein, the second result represents the coarse-grained inter-series dependency relationship, and coarse-grained means taking a single historical time series as the granularity.

[0123] The prediction module 604 performs a fusion prediction based on the first result and the second result to obtain multiple prediction time series, each prediction time series including feature data of a variable at multiple future time points.

[0124] It should be noted that the prediction device provided in the above embodiments is only illustrated by the division of the above functional modules. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the computer device can be divided into different functional modules to complete all or part of the functions described above. In addition, the prediction device and prediction method embodiments provided in the above embodiments belong to the same concept, and their specific implementation process can be found in the method embodiments, which will not be repeated here.

[0125] This specification provides a computer device for performing the above-described method, which can be implemented as a terminal or a server.

[0126] In some embodiments, Figure 7 This is a schematic diagram of the structure of a computer device provided in an embodiment of this application. The computer device 700 includes a memory 701 and a processor 702, wherein the memory 701 stores executable program code 7011, and the processor 702 is used to call and execute the executable program code 7011 to perform a time series prediction method.

[0127] In one embodiment, this application also protects an apparatus that may include a memory and a processor, wherein the memory stores executable program code, and the processor is used to call and execute the executable program code to perform a time series prediction method provided in this application.

[0128] It should be understood that the apparatus provided in this embodiment is used to execute the above-described time series prediction method, and therefore can achieve the same effect as the above-described implementation method.

[0129] When using integrated units, the device may include a processing module and a storage module. When applied to a computer, the processing module can be used to control and manage the computer's operations. The storage module can be used to support the computer in executing relevant program code, etc.

[0130] The processing module may be a processor or a controller, which can implement or execute various exemplary logic blocks, modules, and circuits shown in conjunction with the disclosure of this application. The processor may also be a combination of functions that implement computing capabilities, such as a combination of one or more microprocessors, a combination of digital signal processing (DSP) and a microprocessor, etc., and the storage module may be a memory.

[0131] In addition, the apparatus provided in the embodiments of this application may specifically be a chip, component or module. The chip may include a connected processor and a memory. The memory is used to store instructions. When the processor calls and executes the instructions, the chip can execute a time series prediction method provided in the above embodiments.

[0132] This embodiment also provides a computer-readable storage medium storing computer program code. When the computer program code is run on a computer, the computer executes the aforementioned method steps to implement the time series prediction method provided in the above embodiment.

[0133] This embodiment also provides a computer program product that, when run on a computer, causes the computer to perform the aforementioned steps to implement a time series prediction method provided in the above embodiment.

[0134] In this embodiment, the device, computer-readable storage medium, computer program product, or chip are all used to execute the corresponding methods provided above. Therefore, the beneficial effects they can achieve can be referred to the beneficial effects in the corresponding methods provided above, and will not be repeated here.

[0135] Through the above description of the embodiments, those skilled in the art will understand that, for the sake of convenience and brevity, only the division of the above functional modules is used as an example. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0136] In the embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another device, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0137] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A method of time series forecasting, characterized by, The method comprises: obtaining a plurality of historical time series, each historical time series comprising feature data of a variable at a plurality of historical time points; inputting the plurality of historical time series into a time series branch to obtain a first result output by the time series branch, the first result representing a dependency relationship between different time segments within the same historical time series and a fine-grained inter-sequence dependency relationship, the fine-grained referring to a single time segment as a granularity; inputting the plurality of historical time series into a multivariate branch to obtain a second result output by the multivariate branch, the second result representing a coarse-grained inter-sequence dependency relationship, the coarse-grained referring to a single historical time series as a granularity; performing fusion prediction according to the first result and the second result to obtain a plurality of predicted time series, each predicted time series comprising feature data of a variable at a plurality of future time points.

2. The method of claim 1, wherein, The time series branch comprises a preprocessing module, a first self-attention processing module, a first cross-attention processing module, and a second cross-attention processing module. The inputting the plurality of historical time series into the time series branch to obtain the first result output by the time series branch comprises: inputting the plurality of historical time series into the preprocessing module to preprocess each historical time series to obtain a time segment representation of each historical time series, the time segment representation comprising embedding representations corresponding to a plurality of time segments within the historical time series; inputting the time segment representation of the historical time series into the first self-attention processing module to obtain a first context-aware embedding sequence, the first context-aware embedding sequence fusing the dependency relationship between different time segments within the historical time series; inputting an adaptive aggregation result and the first context-aware embedding sequence into the first cross-attention processing module to obtain a second context-aware embedding sequence, the adaptive aggregation result being a variable interdependence matrix calculated in the multivariate branch in the variable dimension, the second context-aware embedding sequence fusing the fine-grained inter-sequence dependency relationship by weighting and aggregating the first context-aware embedding of each historical time series; inputting the second context-aware embedding sequence and a third context-aware embedding sequence into the second cross-attention processing module to obtain the first result output by the time series branch, the third context-aware embedding sequence being obtained based on the multivariate branch, the third context-aware embedding sequence fusing the coarse-grained inter-sequence dependency relationship.

3. The method of claim 2, wherein, The preprocessing module comprises a standardization processing module, a multi-scale splitting module, and a first linear projection module, and the inputting the plurality of historical time series into the preprocessing module to preprocess each historical time series to obtain a time segment representation of each historical time series comprises: inputting each historical time series into the standardization processing module to apply reversible instance normalization to each historical time series to obtain a standardized historical time series; inputting the standardized historical time series into the multi-scale decomposition module, performing multi-scale segmentation on the standardized historical time series, and converting the standardized historical time series into a plurality of time segments with position embedding; inputting the plurality of time segments with position embedding into the first linear projection module, mapping the plurality of time segments with position embedding into embedded representations of a preset dimension through a first linear projection function, and obtaining time segment representations of each of the historical time series at each scale.

4. The method of claim 2, wherein, The first self-attention processing module includes a first self-attention mechanism layer and a first post-processing layer. The time segment representations of the historical time series are input into the first self-attention processing module to obtain a first context-aware embedding sequence, including: inputting the time segment representations of the historical time series into the first self-attention mechanism layer to obtain a first intermediate representation output by the first self-attention mechanism layer, the first intermediate representation representing the dependency between different time segments within the same historical time series; inputting the first intermediate representation and the time segment representations of the historical time series into the first post-processing layer, performing residual connection on the first intermediate representation and the time segment representations of the historical time series through the first post-processing layer, performing layer normalization on the result of the residual connection, performing nonlinear transformation on the normalized result through a feedforward network layer, and performing layer normalization again on the result of the nonlinear transformation to obtain a first context-aware embedding sequence.

5. The method of claim 2, wherein, The first cross-attention processing module includes a first cross-attention layer and a second post-processing layer. The adaptive aggregation result and the first context-aware embedding sequence are input into the first cross-attention processing module to obtain a second context-aware embedding sequence, including: inputting the adaptive aggregation result and the first context-aware embedding sequence into the first cross-attention layer to obtain a second intermediate representation output by the first cross-attention layer, the second intermediate representation representing the fine-grained inter-sequence dependency relationship; inputting the second intermediate representation and the first context-aware embedding sequence into the second post-processing layer, performing residual connection on the second intermediate representation and the first context-aware embedding sequence through the second post-processing layer, performing normalization on the result of the residual connection, performing nonlinear transformation on the normalized result through a feedforward network layer, and performing layer normalization again on the result of the nonlinear transformation to obtain a second context-aware embedding sequence.

6. The method of claim 2, wherein, The second cross-attention processing module includes a second cross-attention layer and a third post-processing layer. The second context-aware embedding sequence and the third context-aware embedding sequence are input into the second cross-attention processing module to obtain a first result output by the time sequence branch, including: inputting the second context-aware embedding sequence and the third context-aware embedding sequence into the second cross-attention layer to obtain a third intermediate representation; inputting the third intermediate representation and the second context-aware embedding sequence into the third post-processing layer, performing residual connection on the third intermediate representation and the second context-aware embedding sequence through the third post-processing layer, performing layer normalization on a result of the residual connection, performing nonlinear transformation on a result of the layer normalization through a feedforward network layer, and performing layer normalization again on a result of the nonlinear transformation to obtain a first result.

7. The method of claim 1, wherein, The multivariate branch includes a second linear projection module, a second self-attention processing module, and a third cross-attention processing module, and the inputting of the plurality of historical time sequences into the multivariate branch to obtain a second result output by the multivariate branch includes: inputting the plurality of historical time sequences into the second linear projection module to convert the plurality of historical time sequences through a second linear projection function to obtain an embedding representation of the plurality of historical time sequences; inputting the embedding representation of the plurality of historical time sequences into the second self-attention processing module to obtain a third context-aware embedding sequence output by the second self-attention processing module, the third context-aware embedding sequence fusing the coarse-grained inter-sequence dependency relationship; inputting the third context-aware embedding sequence and a second context-aware embedding sequence into the third cross-attention processing module to obtain the second result, the second context-aware embedding sequence being obtained based on the time series branch, and the second context-aware embedding sequence fusing the fine-grained inter-sequence dependency relationship.

8. The method of claim 7, wherein, The second self-attention processing module includes a second self-attention layer and a fourth post-processing layer, and the inputting of the embedding representation of the plurality of historical time sequences into the second self-attention processing module to obtain the third context-aware embedding sequence output by the second self-attention processing module includes: inputting the embedding representation of the plurality of historical time sequences into the second self-attention layer to obtain a fourth intermediate representation, the fourth intermediate representation representing the coarse-grained inter-sequence dependency relationship; inputting the fourth intermediate representation and the embedding representation of the plurality of historical time sequences into the fourth post-processing layer, performing residual connection on the fourth intermediate representation and the embedding representation of the plurality of historical time sequences, performing layer normalization on a result of the residual connection, performing nonlinear transformation on a result of the layer normalization through a feedforward network layer, and performing layer normalization again on a result of the nonlinear transformation to obtain the third context-aware embedding sequence.

9. The method of claim 7, wherein, The third cross-attention processing module includes a third cross-attention layer and a fifth post-processing layer, and the inputting of the third context-aware embedding sequence and the second context-aware embedding sequence into the third cross-attention processing module to obtain the second result output by the third cross-attention processing module includes: inputting the third context-aware embedding sequence and the second context-aware embedding sequence into the third cross-attention layer to obtain a fifth intermediate representation; inputting the fifth intermediate representation and the third context-aware embedding sequence into the fifth post-processing layer, performing residual connection on the fifth intermediate representation and the third context-aware embedding sequence, performing layer normalization on the result of the residual connection, performing nonlinear transformation on the normalized result by a feedforward network layer, performing layer normalization again on the result of the nonlinear transformation, and obtaining the second result.

10. A computer device, comprising: The computer device comprises one or more processors and one or more memories, and at least one computer program is stored in the one or more memories, and the computer program is loaded and executed by the one or more processors to implement the method in any one of claims 1 to 9.

11. A computer readable storage medium, characterized in that, At least one computer program is stored in the computer readable storage medium, and the computer program is loaded and executed by the processor to implement the method in any one of claims 1 to 9.

Images