Long-term multivariable time sequence prediction method and device based on multistage decomposition and Mamba, terminal, medium and product

Abstract

The invention provides a long-term multivariable time sequence prediction method and device based on multistage decomposition and Mamba, a terminal, a medium and a product. The method comprises the following steps: acquiring a target multivariable time sequence; and inputting the target multivariable time sequence into a trained prediction model based on multilevel decomposition and Mamba to obtain a predicted long-term time sequence. The prediction model based on multi-level decomposition and Mama is composed of a preprocessing module, an adaptive time channel decomposition module, a multi-level decomposition module of mixed frequency and time domain, a multi-scale Mama modeling module and a prediction generation module. According to the method, deep mining and efficient combined modeling of the multivariable time sequence are realized, the prediction precision is improved, the computing resources are saved, and the system robustness is enhanced.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to a method, device, terminal, medium and product for long-term multivariate time series prediction based on multi-level decomposition and Mamba. Background Art

[0002] Time series data is ubiquitous in the real world. As a core task in processing time series data, time series forecasting has attracted widespread attention and has broad application value in areas such as climate change prediction, financial market analysis, energy consumption modeling, communication network traffic monitoring, and traffic flow forecasting. With the increasing dimensionality of multivariate time series data, the shift toward long-term forecasting requirements, and the increasing complexity of data nonlinearity and noise fluctuations, traditional time series analysis methods face significant challenges.

[0003] The development of deep learning technology has driven the advancement of time series forecasting. For example, models based on recurrent neural networks (RNNs), with their inherent advantages in sequence modeling due to their cyclic structure, have become a key deep learning approach for time series modeling. However, their cyclic structure can be affected by modeling long-term dependencies, leading to vanishing gradients and making it difficult to model multivariate time series. Models based on convolutional neural networks (CNNs) primarily extract local features, but convolutions are limited by their limited window size and cannot model long-term dependencies for global modeling. Multilayer perceptrons (MLPs) have been widely used in time series modeling due to their simple structure, but their ability to model long-term relationships is limited. This capability can only be improved by combining them with multi-scale information extraction and other techniques. Despite their excellent performance, the Transformer model's quadratic computational complexity still poses efficiency challenges when processing long time series. State-space models, such as Mamba, demonstrate the potential for efficient processing of long sequences through linear complexity and hardware-aware algorithms. However, existing Mamba and its variants (such as SMamba) are not optimized for the multi-scale dynamics of time series and suffer from feature confusion when directly applied to complex sequences.

[0004] Time series are generally a mixture of seasonal, cyclical, and randomly fluctuating data. The complex interwoven temporal dynamics between the data make the prediction results full of uncertainty, and time series decomposition techniques are needed to process the original series. Existing time series decomposition methods (such as STL) rely on fixed rules to separate trend and seasonal components, which makes it difficult to adapt to the dynamic changes of non-stationary series. Models using frequency domain decomposition methods (such as Autoformer and FEDformer) can extract global features through Fourier transform, but they are not deeply integrated with time domain information. Independent processing of frequency domain signals is prone to introducing high-frequency noise. Existing research focuses on single-scale feature extraction and lacks a systematic design for joint time-frequency analysis. Summary of the Invention

[0005] In view of the shortcomings of the prior art described above, the purpose of this application is to provide a long-term multivariate time series prediction method, device, terminal, medium and product based on multi-level decomposition and Mamba, which is used to solve the problems of insufficient dynamic decomposition capability, limited computational efficiency and insufficient multi-scale feature fusion in the prior art in high-dimensional multivariate time series prediction.

[0006] To achieve the above-mentioned purpose and other related purposes, the first aspect of the present application provides a long-term multivariate time series prediction method based on multi-level decomposition and Mamba, comprising: obtaining a target multivariate time series; inputting the target multivariate time series into a trained prediction model based on multi-level decomposition and Mamba to obtain a predicted long-term time series; wherein the prediction model based on multi-level decomposition and Mamba is composed of a preprocessing module, an adaptive time channel decomposition module, a mixed frequency and time domain multi-level decomposition module, a multi-scale Mamba modeling module and a prediction generation module; the preprocessing module, the adaptive time channel decomposition module, the mixed frequency and time domain multi-level decomposition module and the multi-scale Mamba modeling module are connected in sequence; and the prediction generation module is respectively connected to the adaptive time channel decomposition module and the multi-scale Mamba modeling module.

[0007] In some embodiments of the first aspect of the present application, the processing of the preprocessing module includes: performing missing value filling processing, noise suppression processing and normalization processing on the target multivariate time series to obtain a preprocessed multivariate time series.

[0008] In some embodiments of the first aspect of the present application, the processing of the adaptive time channel decomposition module includes: mapping the preprocessed multivariate time series to a low-dimensional space to obtain low-dimensional space data; using a multi-head attention mechanism and an activation function to extract an initial global trend component from the low-dimensional space data after random discarding; based on the initial global trend component, separating the initial seasonal component from the low-dimensional space data; projecting the initial seasonal component and the initial global trend component respectively to obtain the final seasonal component and global trend component.

[0009] In some embodiments of the first aspect of the present application, the processing of the mixed frequency and time domain multi-level decomposition module includes: using discrete wavelet transform to extract high-frequency components and low-frequency components from the final seasonal components; performing root mean square normalization on the final seasonal components to obtain time domain information; and combining the high-frequency components, low-frequency components and time domain information to form mixed time-frequency decomposition information.

[0010] In some embodiments of the first aspect of the present application, the multi-scale Mamba modeling module is composed of multiple Karma modules connected in sequence, which is used to generate a multi-scale modeling result based on the mixed time-frequency decomposition information; wherein each Karma module is composed of a high-frequency unit, a time domain unit and a low-frequency unit; the high-frequency unit, the time domain unit and the low-frequency unit all adopt the Mamba model.

[0011] In some embodiments of the first aspect of the present application, the processing of the prediction generation module includes: restoring the multi-scale modeling results to time-domain seasonal forecast data through inverse discrete wavelet transform; performing global Mamba modeling and global projection on the final global trend component in turn to obtain trend forecast data; performing linear weighted fusion on the time-domain seasonal forecast data and the trend forecast data, and performing output projection and denormalization processing on the weighted fused data in turn to obtain a predicted long-term time series.

[0012] To achieve the above-mentioned purpose and other related purposes, the second aspect of the present application provides a long-term multivariate time series prediction device based on multi-level decomposition and Mamba, comprising: an acquisition module for acquiring a target multivariate time series; a prediction module for inputting the target multivariate time series into a trained prediction model based on multi-level decomposition and Mamba to obtain a predicted long-term time series; wherein the prediction model based on multi-level decomposition and Mamba is composed of a preprocessing module, an adaptive time channel decomposition module, a multi-level decomposition module of mixed frequency and time domain, a multi-scale Mamba modeling module and a prediction generation module connected in sequence; the preprocessing module, the adaptive time channel decomposition module, the multi-level decomposition module of mixed frequency and time domain and the multi-scale Mamba modeling module are connected in sequence; the prediction generation module is connected to the adaptive time channel decomposition module and the multi-scale Mamba modeling module respectively.

[0013] To achieve the above-mentioned purpose and other related purposes, the third aspect of the present application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the long-term multivariate time series prediction method based on multi-level decomposition and Mamba.

[0014] To achieve the above-mentioned objectives and other related objectives, the fourth aspect of the present application provides a computer program product, which includes computer program code. When the computer program code is run on a computer, the computer implements the long-term multivariate time series prediction method based on multi-level decomposition and Mamba.

[0015] To achieve the above-mentioned purpose and other related purposes, the fifth aspect of the present application provides an electronic terminal, including a memory, a processor and a computer program stored in the memory; the processor executes the computer program to implement the long-term multivariate time series prediction method based on multi-level decomposition and Mamba.

[0016] As described above, the long-term multivariate time series prediction method, device, terminal, medium, and product based on multi-level decomposition and Mamba of the present application have the following beneficial effects:

[0017] This application achieves deep mining and efficient joint modeling of multivariate time series, improves prediction accuracy, saves computing resources, and enhances system robustness. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 Shown is a flowchart of a long-term multivariate time series prediction method based on multi-level decomposition and Mamba in one embodiment of the present application.

[0019] Figure 2 Shown is a structural diagram of a prediction model based on multi-level decomposition and Mamba in one embodiment of the present application.

[0020] Figure 3 Shown is a structural diagram of an adaptive time channel decomposition module in one embodiment of the present application.

[0021] Figure 4 Shown is a structural diagram of a mixed frequency and time domain multi-level decomposition module and a multi-scale Mamba modeling module in one embodiment of the present application.

[0022] Figure 5 Shown is a structural diagram of the Mamba model in one embodiment of the present application.

[0023] Figure 6 Shown is a structural diagram of a time domain unit in one embodiment of the present application.

[0024] Figure 7 Shown are visualization results of the prediction model in a specific embodiment of the present application on four real data sets.

[0025] Figure 8 Shown is a comparison chart of the average experimental results of the prediction model in a specific embodiment of the present application on 7 real-world data sets compared with 7 existing methods.

[0026] Figure 9 Shown is a schematic diagram of ablation experiment results in a specific embodiment of the present application.

[0027] Figure 10Shown is a schematic diagram of generalization experimental results for different input lengths in a specific embodiment of the present application.

[0028] Figure 11 Shown is a performance and effect comparison chart of the prediction model in a specific embodiment of the present application.

[0029] Figure 12 Shown is a schematic block diagram of a long-term multivariate time series prediction device based on multi-level decomposition and Mamba in one embodiment of the present application.

[0030] Figure 13 Shown is a structural schematic diagram of an electronic terminal in one embodiment of the present application. DETAILED DESCRIPTION

[0031] The following describes the embodiments of the present application through specific examples. Those skilled in the art can easily understand the other advantages and effects of the present application from the content disclosed in this specification. The present application can also be implemented or applied through other different specific embodiments. The details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of the present application. It should be noted that the following embodiments and features in the embodiments can be combined with each other unless they conflict.

[0032] In the embodiments of this application, terms such as "first" and "second" are used to distinguish between identical or similar items with substantially the same functions and effects. Those skilled in the art will understand that terms such as "first" and "second" do not limit the quantity or order of execution, and that terms such as "first" and "second" do not necessarily define differences.

[0033] It should be noted that in the embodiments of this application, words such as "exemplary" or "for example" represent examples, illustrations, or descriptions. Any embodiment or design described in this application as "exemplary" or "for example" should not be interpreted as being preferred or advantageous over other embodiments or designs. Rather, the use of words such as "exemplary" or "for example" is intended to present the relevant concepts in a concrete manner.

[0034] In the embodiments of the present application, "at least one" refers to one or more, and "more" refers to two or more. "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can represent: the existence of A alone, the existence of A and B at the same time, and the existence of B alone, where A and B can be singular or plural. The character " / " generally indicates that the previous and next associated objects are in an "or" relationship. "At least one of the following items" or similar expressions refers to any combination of these items, including any combination of single items or plural items. For example, at least one of a, b or c can represent: a, b, c, ab, ac, bc or abc, where a, b, c can be single or multiple.

[0035] To facilitate understanding of the embodiments of this application, first Figure 1 Detailed description. Figure 1 The flowchart of a long-term multivariate time series prediction method based on multi-level decomposition and Mamba in an embodiment of the present invention is shown. The long-term multivariate time series prediction method based on multi-level decomposition and Mamba in this embodiment mainly includes the following steps:

[0036] Step S11: Obtain the target multivariate time series.

[0037] It should be understood that multivariate time series (MTS) refers to time series data containing multiple variables that are temporally correlated and typically generated by the same system. The dimensions of target multivariate data can range from tens to hundreds.

[0038] In a specific implementation, the types of target multivariate time series include but are not limited to power load time series, financial market time series, traffic flow time series, etc.

[0039] Step S12: Input the target multivariate time series into the trained prediction model based on multi-level decomposition and Mamba to obtain the predicted long-term time series.

[0040] In one embodiment, if Figure 2 As shown in FIG, the prediction model based on multi-level decomposition and Mamba consists of a preprocessing module, an adaptive time channel decomposition module (ATCD module), a mixed frequency and time domain multi-level decomposition module (HFTD module), a multi-scale Mamba modeling module, and a prediction generation module; wherein the preprocessing module, the adaptive time channel decomposition module, the mixed frequency and time domain multi-level decomposition module, and the multi-scale Mamba modeling module are connected in sequence; and the prediction generation module is connected to the adaptive time channel decomposition module and the multi-scale Mamba modeling module respectively.

[0041] In one embodiment, the processing of the preprocessing module includes: performing missing value filling processing, noise suppression processing, and normalization processing on the target multivariate time series to obtain a preprocessed multivariate time series.

[0042] It should be understood that the methods of filling missing values ​​include but are not limited to mean filling, interpolation, etc. Those skilled in the art can choose a suitable method of filling missing values ​​according to actual needs, and this embodiment does not limit this. Noise suppression refers to the use of various technical means to reduce or eliminate unnecessary and interfering information (noise) in the data during data analysis and processing, thereby improving data quality, enhancing data availability and the accuracy of analysis results. Those skilled in the art can choose any existing method for noise suppression, such as low-pass filtering, as needed, and this embodiment does not limit this. Data normalization is a commonly used method in data preprocessing, the purpose of which is to scale the data so that it falls within a small range (usually [0,1] or [-1,1]), thereby improving the consistency of the data and the performance of the model. Those skilled in the art can choose any existing method for normalization, and this embodiment does not limit this.

[0043] In one embodiment, if Figure 3 As shown, the processing of the adaptive time channel decomposition module includes: mapping the preprocessed multivariate time series to a low-dimensional space to obtain low-dimensional space data; using a multi-head attention mechanism and an activation function to extract an initial global trend component from the low-dimensional space data after random discarding; based on the initial global trend component, separating the initial seasonal component from the low-dimensional space data; projecting the initial seasonal component and the initial global trend component respectively to obtain the final seasonal component and global trend component.

[0044] Specifically, refer to the following formula 1, the preprocessed multivariate time series pass Map to low-dimensional space to obtain low-dimensional space data And for low-dimensional space data Perform random discarding to obtain data after random discarding

[0045]

[0046] Among them, Dropout(*)=r⊙*,r~Bernoulli(p), Dropout(*) is used to randomly discard a part of the data to reduce overfitting, and di represents the dimension of the low-dimensional space.

[0047] It should be understood that in probability theory and statistics, the "Bernoulli" distribution is a type of discrete probability distribution. Mapping multivariate data to a low-dimensional space is a method of data dimensionality reduction, the purpose of which is to convert high-dimensional data into low-dimensional data. Those skilled in the art may choose any existing dimensionality reduction method for dimensionality reduction, and this application is not limited thereto.

[0048] Furthermore, referring to the following formulas 2 and 3, the multi-head attention mechanism is used to randomly discard the data. Extract and obtain the extraction results, and refine the extraction results through the activation function SiLU(*) to obtain the initial global trend component

[0049]

[0050] Among them, MHA(*) represents the multi-head attention mechanism, h represents the number of attention heads, and d n represents the dimension of the embedding vector, corresponding to As a scaling factor. The normalized weight is obtained by softmax(*) operation, Q h ,K h ,V h Is input Through the weight matrix W Q ,W K ,W V Calculated, W O is the final linear transformation weight matrix, and Concat(*) represents concatenation.

[0051] Furthermore, referring to the following formula 4, using the residual, and based on the initial global trend component From low-dimensional space data Separate the initial seasonal component

[0052]

[0053] Further, referring to the following formula 5, through the linear mapping function of the global trend component The initial global trend component Project to different scales to obtain the final global trend component

[0054]

[0055] Refer to the following formula 6, through the linear mapping function of the seasonal component The initial seasonal component Project to different scales to obtain the final seasonal components

[0056]

[0057] It should be understood that in this embodiment, terms such as "global trend component" and "seasonal component" are used to distinguish linear mapping functions with essentially the same functionality and effect, and that the terms "global trend component" and "seasonal component" are not necessarily different. A linear mapping function is a mathematical function that maps elements in one vector space to another while maintaining vector addition and scalar multiplication operations.

[0058] It should be noted that through the adaptive time channel decomposition module, the global trend and local periodic changes in the data can be extracted, the trend and seasonal components can be effectively separated, and the accuracy of subsequent predictions can be enhanced.

[0059] In one embodiment, if Figure 4 As shown, the processing of the mixed frequency and time domain multi-level decomposition module includes: using discrete wavelet transform to extract high-frequency components and low-frequency components from the final seasonal components; performing root mean square normalization on the final seasonal components to obtain time domain information; and combining the high-frequency components, low-frequency components and time domain information to form mixed time-frequency decomposition information.

[0060] Specifically, referring to the following formulas 7, 8, and 9, the final seasonal component is transformed by discrete wavelet transform (DWT) Perform multi-scale decomposition to obtain high-frequency components and low-frequency components The wavelet basis function can be DB4 or Haar, or other types of wavelet basis functions, which are not limited in this embodiment. The number of decomposition layers of discrete wavelet transform can be 1 to 3, or other numbers, which are not limited in this embodiment.

[0061]

[0062] Among them, d represents the feature channel index, n represents the time step index, the sequence is wavelet decomposed one by one according to the dimension, and k represents the displacement parameter after discrete wavelet transform, which depends on the input (the final seasonal component ), the scale j corresponds to different decomposition levels. The basis function φ used to capture low-frequency components and high-frequency components j,k [n] and ψ j,k [n] Refer to formulas 10 and 11 below:

[0063]

[0064] Furthermore, referring to the following formula 12, the final seasonal component Perform root mean square normalization RMSNorm(*) to obtain time domain information

[0065]

[0066] Here, RMSNorm(*) represents root mean square normalization. It should be understood that RMSNorm (Root Mean Square Normalization) is a normalization technique used in deep learning models that aims to improve computational efficiency by simplifying the normalization process while maintaining model performance.

[0067] Furthermore, the high frequency components low-frequency component and time domain information It constitutes the mixed time-frequency decomposition information.

[0068] It should be noted that the multi-level decomposition module of mixed frequency and time domains fully utilizes the complementary advantages of the frequency domain and time domain, which not only captures the rapid fluctuation characteristics of the data, but also retains the periodicity and long-term trend information, effectively improving the prediction model's ability to express complex time series data.

[0069] In one embodiment, if Figure 4 As shown, the multi-scale Mamba modeling module is composed of N Karma modules connected in sequence, which is used to generate a multi-scale modeling result based on the mixed time-frequency decomposition information; wherein N is a positive integer; each Karma module is composed of a high-frequency unit, a time-domain unit and a low-frequency unit; the high-frequency unit, the time-domain unit and the low-frequency unit all adopt the Mamba model.

[0070] In one embodiment, the multi-scale Mamba modeling module performs processing with reference to the following formulas 13, 14, and 15:

[0071]

[0072] in, Represents the high-frequency unit, Represents the low-frequency unit, Represents the Mamba model in time domain unit, i∈[1,N].

[0073] Specifically, the output of each Karma module in the multi-scale Mamba modeling module serves as the input to the next Karma module. The high-frequency units of each Karma module in the multi-scale Mamba modeling module are connected sequentially, the low-frequency units of each Karma module in the multi-scale Mamba modeling module are connected sequentially, and the time-domain units of each Karma module in the multi-scale Mamba modeling module are connected sequentially.

[0074] The mixed time-frequency decomposition information serves as the input to the multi-scale Mamba modeling module, and the output of the multi-scale Mamba modeling module is the modeled high-frequency component, low-frequency component, and time domain information. The high-frequency component in the mixed time-frequency decomposition information serves as the input to the first high-frequency unit, the low-frequency component in the mixed time-frequency decomposition information serves as the input to the first low-frequency unit, and the time domain information in the mixed time-frequency decomposition information serves as the input to the first time domain unit. The modeled high-frequency component serves as the output of the last high-frequency unit, the modeled low-frequency component serves as the output of the last low-frequency unit, and the modeled time domain information serves as the output of the last time domain unit. The modeled high-frequency component, low-frequency component, and time domain information constitute the multi-scale modeling results.

[0075] The high-frequency unit and low-frequency unit adopt the Mamba model. The specific structure of the Mamba model is as follows: Figure 5 As shown in Figure 1, the input is projected and then processed by the silu activation function to obtain the processing result of the first branch. Simultaneously, the input is processed sequentially by convolution, the silu activation function, and the selective state-space mechanism to obtain the processing result of the second branch. The processing results of the first and second branches are added together, and the added result is projected and output. It should be noted that the specific settings of the Mamba model can refer to the existing Mamba model and will not be repeated here.

[0076] Time domain unit such as Figure 6 As shown in Figure 1, the feature channel dimension of the time-domain unit input is inverted. This inverted input, along with the RMS-normalized time-domain unit input, is then fed into the Mamba model. It should be noted that the structure of the Mamba model for the time-domain unit is identical to that of the Mamba model mentioned above and will not be further described here. The output of the Mamba model is added to the input of the time-domain unit to obtain the output of the time-domain unit.

[0077] The selective state-space mechanism uses the following discrete state-space equations:

[0078]

[0079] Among them, the matrix Initialization is performed using a specific parameterization method and dynamically adjusted based on the data to achieve modeling of long-term temporal dependencies. The specific initialization method can refer to the initialization method of the existing Mamba model and will not be repeated here.

[0080] It should be noted that the multi-scale Mamba modeling module combines local details with global information through a layer-by-layer approach. The output of each layer serves as the input to the next layer, ultimately achieving a multi-scale joint modeling result. The module's internal design utilizes a selective state space mechanism, ensuring that computational complexity only increases linearly with sequence length, making it suitable for real-time prediction of extremely long sequences. After processing by the multi-scale Mamba modeling module, it fully captures the multi-level information hidden in the data, achieving a deep fusion of frequency and time domain information of seasonal components.

[0081] In one embodiment, if Figure 2 As shown in the figure, the processing of the prediction generation module includes: restoring the multi-scale modeling results to time-domain seasonal forecast data through inverse discrete wavelet transform; performing global Mamba modeling and global projection on the final global trend component in turn; performing linear weighted fusion on the time-domain seasonal forecast data and trend forecast data, and performing output projection and denormalization on the weighted fused data in turn to obtain the predicted long-term time series.

[0082] Specifically, referring to the following formula 16, the multi-scale modeling results are subjected to inverse wavelet transform (IDWT) to obtain the time domain seasonal prediction data The inverse wavelet transform uses the wavelet basis functions shown in formulas 10 and 11:

[0083]

[0084] Further, referring to the following formula 17, the final global trend component Input into the global Mamba module and pass through the global linear mapping function Perform global projection on the output of the global Mamba module to obtain trend prediction data

[0085]

[0086] in, It represents the global Mamba module. The specific structure of the global Mamba module is the same as that of the Mamba model in the above embodiment, and will not be repeated here.

[0087] Furthermore, referring to the following formula 18, linear weighted fusion is performed on the time-domain seasonal forecast data and the trend forecast data, and the output linear mapping function is used to output the weighted fusion result. The output projection result is denormalized to obtain the predicted long-term time series:

[0088]

[0089] Among them, DeNorm(*) means denormalization, represents the linear projection function of the output, Represents the long-term time series for forecasting.

[0090] It should be understood that in this embodiment, words such as "global" and "output" are used to distinguish linear mapping functions with basically the same functions and effects, and words such as "global" and "output" do not necessarily limit them to be different. A linear mapping function is a mathematical function that maps elements in a vector space to another vector space while maintaining vector addition and scalar multiplication operations. Weighted fusion is a method of assigning weights to different data sources or models, and then weighted summing or combining these data or results. Inverse Normalization is the process of restoring normalized data to its original range. The time length of the predicted long-term time series is greater than or equal to the time length of the target multivariate time series. The scale of the predicted long-term time series other than time is the same as that of the target multivariate time series.

[0091] In one embodiment, the obtained time series dataset is used to train an untrained prediction model based on multi-level decomposition and Mamba; wherein the time series dataset can be any one of the following: ECL, Weather, and ETTm1.

[0092] In one embodiment, to ensure the prediction accuracy of the model, a loss function that integrates time domain and frequency domain errors is used during model training, and the Adam optimizer, learning rate decay, and early stopping strategy are used to optimize the model parameters until convergence, thereby obtaining a trained prediction model based on multi-level decomposition and Mamba.

[0093] Specifically, the loss function of the integrated time domain and frequency domain errors is As shown in Formula 19:

[0094]

[0095] The parameter α is selected to a suitable value (such as 0.2 or 0.5) according to the data characteristics.

[0096] In one embodiment, if Figure 7 as well as Figure 8 As shown, the prediction model based on multi-level decomposition and Mamba of the present invention is compared with existing methods (such as iTransformer, Autoformer, SMamba, etc.) on multiple real data sets (such as ECL, Traffic, Weather, ETTm1, etc.), and the main evaluation indicators are mean square error (MSE) and mean absolute error (MAE). In the ECL data set, when the prediction window is set to 96, the MSE of the model of the present invention is 0.136 and the MAE is 0.231. Compared with the MSE0.149 and MAE 0.241 of the iTransformer model and the MSE0.168 and MAE 0.272 of the Autoformer model, the prediction accuracy of the model of the present invention is improved by about 8% to 12%, respectively. In addition, the prediction model based on multi-level decomposition and Mamba adopted by the present invention reduces the overall model parameter amount by about 65% compared with the traditional Transformer model, greatly reducing the computational complexity and energy consumption requirements while ensuring the prediction performance, and can be generalized to the prediction length of longer sequences.

[0097] In a specific embodiment, an ablation experiment was conducted to verify the improvement effect of the hybrid frequency and time domain multi-level decomposition (HFTD) module and the adaptive time channel decomposition (ATCD) module on the model performance. Figure 9 As shown in the data, in the ECL dataset, after introducing the HFTD module into the model architecture, the forecast error (MSE and MAE) decreased by 11.83% and 31.97%, respectively. This fully demonstrates the significant role of the HFTD module in extracting frequency and time domain features and improving model robustness. After removing the ATCD module, the MSE and MAE of the model increased by approximately 9.2% and 14.5%, respectively, indicating that the ATCD module plays a key role in capturing global trend information and accurately separating seasonal components.

[0098] In one embodiment, Figure 10 As shown, the generalization experimental structure of the prediction model (KARMA) based on multi-level decomposition and Mamba adopted by the present invention and other models (such as iTransformer, Autoformer, SMamba, etc.) under different input time series lengths is demonstrated.

[0099] In one embodiment, Figure 11The figure shows the model efficiency comparison of the prediction model (KARMA) based on multi-level decomposition and Mamba adopted by the present invention and other models (such as iTransformer, Autoformer, SMamba, etc.) on the ECL dataset when the input is 720 and the prediction is 720, as well as the training efficiency comparison corresponding to different input and output lengths.

[0100] In order to better illustrate the application of the prediction model based on multi-level decomposition and Mamba of the present invention, two specific embodiments are first provided:

[0101] Embodiment 1: A financial market data forecast.

[0102] This embodiment uses daily trading data as the input of a prediction model based on multi-level decomposition and Mamba. Since financial data usually has high volatility and noise, the method of the present invention can effectively separate the market trend from short-term fluctuations. In the data preprocessing part, the daily trading data is normalized and the noise is smoothed by the median filter method; in the ATCD module, the multi-head attention mechanism is used to extract the global market trend and separate the short-term cyclical fluctuations; the HFTD module uses discrete wavelet transform to extract high-frequency data to reflect short-term violent fluctuations and low-frequency data to capture long-term cyclical changes, while obtaining time domain information; the multi-scale Mamba modeling module deeply fuses the information of each layer and captures the implicit nonlinear relationship in the data through state space modeling; finally, the prediction generation module fuses the information of each component and outputs the future market trend forecast. Experimental results show that this method not only outperforms existing methods in mean square error (MSE) and mean absolute error (MAE), but also performs more sensitively and accurately in capturing large trends and short-term fluctuations.

[0103] Example 2: Traffic flow prediction.

[0104] For traffic flow monitoring data, this embodiment uses a data set containing hundreds of features and a large sample size. During data preprocessing, traffic data is normalized and missing values ​​are processed; the ATCD module is used to separate seasonal components and overall trends that reflect the periodic changes in traffic flow; the HFTD module uses discrete wavelet transform to separate high-frequency short-term fluctuations and low-frequency long-term changes in traffic data, while extracting time domain features; the multi-scale Mamba modeling module further integrates the information of each component through recursive modeling, effectively capturing local details and global trends in the data; the prediction generation module integrates various information to generate accurate future traffic flow prediction results. Experimental verification shows that this method exhibits high real-time performance and accuracy in long-term prediction, providing reliable data support for intelligent traffic management systems.

[0105] Figure 12Schematic diagram of a long-term multivariate time series prediction device based on multi-level decomposition and Mamba provided by an embodiment of the present application. Figure 12 As shown, the long-term multivariate time series prediction device based on multi-level decomposition and Mamba includes:

[0106] Acquisition module, used to obtain the target multivariate time series;

[0107] A prediction module, configured to input the target multivariate time series into a trained prediction model based on multi-level decomposition and Mamba to obtain a predicted long-term time series;

[0108] Among them, the prediction model based on multi-level decomposition and Mamba is composed of a preprocessing module, an adaptive time channel decomposition module, a multi-level decomposition module of mixed frequency and time domain, a multi-scale Mamba modeling module and a prediction generation module connected in sequence; the preprocessing module, the adaptive time channel decomposition module, the multi-level decomposition module of mixed frequency and time domain and the multi-scale Mamba modeling module are connected in sequence; the prediction generation module is respectively connected to the adaptive time channel decomposition module and the multi-scale Mamba modeling module.

[0109] It should be understood that the specific process of each module executing the above corresponding steps has been described in detail in the above method embodiment, and for the sake of brevity, it will not be repeated here.

[0110] It should also be understood that the division of modules in the embodiments of the present application is illustrative and is merely a logical functional division. In actual implementation, other division methods may be used. Furthermore, the functional modules in the various embodiments of the present application may be integrated into a single processor, or may exist physically separately, or two or more modules may be integrated into a single module. The aforementioned integrated modules may be implemented in the form of hardware or software functional modules.

[0111] In one embodiment, the processing of the preprocessing module includes: performing missing value filling processing, noise suppression processing and normalization processing on the target multivariate time series to obtain a preprocessed multivariate time series.

[0112] In one embodiment, the processing of the adaptive time channel decomposition module includes: mapping the preprocessed multivariate time series to a low-dimensional space to obtain low-dimensional space data; using a multi-head attention mechanism and an activation function to extract an initial global trend component from the low-dimensional space data after random discarding; based on the initial global trend component, separating the initial seasonal component from the low-dimensional space data; projecting the initial seasonal component and the initial global trend component respectively to obtain the final seasonal component and global trend component.

[0113] In one embodiment, the processing of the mixed frequency and time domain multi-level decomposition module includes: using discrete wavelet transform to extract high-frequency components and low-frequency components from the final seasonal components; performing root mean square normalization on the final seasonal components to obtain time domain information; and combining the high-frequency components, low-frequency components and time domain information to form mixed time-frequency decomposition information.

[0114] In one embodiment, the multi-scale Mamba modeling module is composed of multiple Karma modules connected in sequence, which is used to generate a multi-scale modeling result based on the mixed time-frequency decomposition information; wherein each Karma module is composed of a high-frequency unit, a time domain unit and a low-frequency unit; the high-frequency unit, the time domain unit and the low-frequency unit all adopt the Mamba model.

[0115] In one embodiment, the processing of the prediction generation module includes: restoring the multi-scale modeling results into time-domain seasonal forecast data through inverse discrete wavelet transform; performing global Mamba modeling and global projection on the final global trend component in turn to obtain trend forecast data; performing linear weighted fusion on the time-domain seasonal forecast data and the trend forecast data, and performing output projection and denormalization processing on the weighted fused data in turn to obtain a predicted long-term time series.

[0116] Figure 13 This is a schematic block diagram of an electronic terminal provided in an embodiment of the present application. Figure 13 As shown, the electronic terminal includes: at least one processor 1301, a memory 1302, at least one network interface 1303 and a user interface 1305. The various components in the device are coupled together through a bus system 1304. It is understood that the bus system 1304 is used to achieve connection and communication between these components. In addition to including a data bus, the bus system 1304 also includes a power bus, a control bus and a status signal bus. However, for the sake of clarity, Figure 13 Various buses are labeled as bus systems.

[0117] The user interface 1305 may include a display, a keyboard, a mouse, a trackball, a click gun, keys, buttons, a touch pad or a touch screen.

[0118] It will be appreciated that the memory 1302 may be a volatile memory or a non-volatile memory, or may include both volatile and non-volatile memories. Among them, the non-volatile memory may be a read-only memory (ROM) or a programmable read-only memory (PROM), which is used as an external cache. By way of example but not limitation, many forms of RAM are available, such as static random access memory (SRAM) and synchronous static random access memory (SSRAM). The memory described in the embodiments of the present invention is intended to include, but is not limited to, these and any other suitable types of memory.

[0119] The memory 1302 in the embodiment of the present invention is used to store various types of data to support the operation of the electronic terminal 1300. Examples of such data include: any executable program used to operate on the electronic terminal 1300, such as the operating system 13021 and the application 13022; the operating system 13021 includes various system programs, such as the framework layer, the core library layer, the driver layer, etc., for implementing various basic services and processing hardware-based tasks. The application 13022 can include various application programs, such as a media player (Media Player), a browser (Browser), etc., for implementing various application services. The long-term multivariate time series prediction method based on multi-level decomposition and Mamba provided in the embodiment of the present invention can be included in the application 13022.

[0120] The methods disclosed in the above embodiments of the present invention can be applied to or implemented by processor 1301. Processor 1301 may be an integrated circuit chip with signal processing capabilities. During implementation, each step of the above method can be completed by hardware integrated logic circuits in processor 1301 or by software instructions. The above processor 1301 may be a general-purpose processor, a digital signal processor (DSP), or other programmable logic device, discrete gate or transistor logic device, discrete hardware component, etc. Processor 1301 can implement or execute the various methods, steps, and logic block diagrams disclosed in the embodiments of the present invention. The general-purpose processor 1301 may be a microprocessor or any conventional processor. The steps of the accessory optimization method provided in the embodiments of the present invention can be directly implemented and executed by a hardware decoding processor, or by a combination of hardware and software modules in the decoding processor. The software module can be located in a storage medium located in a memory. The processor reads the information in the memory and, in conjunction with its hardware, completes the steps of the above method.

[0121] In an exemplary embodiment, the electronic terminal 1300 may be configured to execute the aforementioned method using one or more application specific integrated circuits (ASICs), DSPs, programmable logic devices (PLDs), or complex programmable logic devices (CPLDs).

[0122] According to the method provided in the embodiment of the present application, the present application also provides a computer program product, which includes: a computer program code, which, when executed on a computer, causes the computer to execute Figure 1 The long-term multivariate time series prediction method based on multi-level decomposition and Mamba in the illustrated embodiment.

[0123] According to the method provided in the embodiment of the present application, the present application also provides a computer-readable storage medium, which stores a program code, and when the program code is run on a computer, the computer executes Figure 1 The long-term multivariate time series prediction method based on multi-level decomposition and Mamba in the illustrated embodiment.

[0124] As used in this specification, the terms "component," "module," "system," and the like are used to represent computer-related entities, hardware, firmware, a combination of hardware and software, software, or software in execution. For example, a component can be, but is not limited to, a process running on a processor, a processor, an object, an executable file, an execution thread, a program, and / or a computer. By way of illustration, both an application running on a computing device and a computing device can be a component. One or more components can reside in a process and / or an execution thread, and a component can be located on a computer and / or distributed between two or more computers. In addition, these components can be executed from various computer-readable media having various data structures stored thereon. Components can communicate, for example, via local and / or remote processes based on signals having one or more data packets (e.g., data from two components interacting with another component on a local system, a distributed system, and / or a network, such as the Internet interacting with other systems via signals).

[0125] Those skilled in the art will appreciate that the various illustrative logical blocks and steps described in conjunction with the embodiments disclosed herein can be implemented using electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians may use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0126] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0127] In the several embodiments provided in this application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are merely schematic. For example, the division of units is only a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0128] Units described as separate components may or may not be physically separate, and components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0129] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0130] In the above embodiments, the functions of each functional unit can be implemented in whole or in part by software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions (programs). When the computer program instructions (program) are loaded and executed on a computer, the process or function according to the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via a wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) method. The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server or data center that includes one or more available media integrated therein. Available media may be magnetic media (e.g., floppy disks, hard disks, tapes), optical media (e.g., high-density digital video discs (DVDs), or semiconductor media (e.g., solid state disks (SSDs)).

[0131] If the function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the various embodiments of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), magnetic disk or optical disk, and other media that can store program codes.

[0132] The above description is merely a specific embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this application should be included in the scope of protection of the present application. Therefore, the scope of protection of the present application should be based on the scope of protection of the claims.

[0133] In summary, the present application provides a method, device, terminal, medium and product for long-term multivariate time series prediction based on multi-level decomposition and Mamba. The method comprises: obtaining a target multivariate time series; inputting the target multivariate time series into a trained prediction model based on multi-level decomposition and Mamba to obtain a predicted long-term time series. The prediction model based on multi-level decomposition and Mamba consists of a preprocessing module, an adaptive time channel decomposition module, a multi-level decomposition module of mixed frequency and time domain, a multi-scale Mamba modeling module and a prediction generation module. The present application realizes deep mining and efficient joint modeling of multivariate time series, improves prediction accuracy, saves computing resources, and enhances system robustness. Therefore, the present application effectively overcomes the various shortcomings in the prior art and has a high industrial utilization value.

[0134] The above embodiments are merely illustrative of the principles and effects of this application and are not intended to limit this application. Anyone skilled in the art may modify or alter the above embodiments without departing from the spirit and scope of this application. Therefore, all equivalent modifications or alterations made by one of ordinary skill in the art without departing from the spirit and technical concepts disclosed in this application shall be covered by the claims of this application.

Claims

1. A long-term multivariate time series forecasting method based on multi-level decomposition and Mamba, characterized by: include: Get the target multivariate time series; Inputting the target multivariate time series into a trained prediction model based on multi-level decomposition and Mamba to obtain a predicted long-term time series; The prediction model based on multi-level decomposition and Mamba consists of a preprocessing module, an adaptive time channel decomposition module, a mixed frequency and time domain multi-level decomposition module, a multi-scale Mamba modeling module and a prediction generation module.

2. The long-term multivariate time series prediction method based on multi-level decomposition and Mamba according to claim 1 is characterized in that: The processing of the preprocessing module includes: performing missing value filling processing, noise suppression processing and normalization processing on the target multivariate time series to obtain a preprocessed multivariate time series.

3. The long-term multivariate time series prediction method based on multi-level decomposition and Mamba according to claim 2 is characterized in that: The processing of the adaptive time channel decomposition module includes: Map the preprocessed multivariate time series to a low-dimensional space to obtain low-dimensional space data; Utilizing a multi-head attention mechanism and an activation function, an initial global trend component is extracted from the low-dimensional spatial data after random dropout processing; based on the initial global trend component, an initial seasonal component is separated from the low-dimensional spatial data; The initial seasonal component and the initial global trend component are projected respectively to obtain the final seasonal component and global trend component.

4. The long-term multivariate time series prediction method based on multi-level decomposition and Mamba according to claim 3 is characterized in that: The processing of the multi-level decomposition module of the mixed frequency and time domain includes: The high-frequency component and the low-frequency component are extracted from the final seasonal component by using discrete wavelet transform; the final seasonal component is normalized by root mean square to obtain the time domain information; The high-frequency component, low-frequency component and time domain information are combined to form mixed time-frequency decomposition information.

5. The long-term multivariate time series prediction method based on multi-level decomposition and Mamba according to claim 4 is characterized in that: The multi-scale Mamba modeling module is composed of multiple Karma modules connected in sequence, and is used to generate a multi-scale modeling result based on the mixed time-frequency decomposition information; wherein each Karma module is composed of a high-frequency unit, a time-domain unit, and a low-frequency unit; the high-frequency unit, the time-domain unit, and the low-frequency unit all adopt the Mamba model.

6. The long-term multivariate time series prediction method based on multi-level decomposition and Mamba according to claim 5, characterized in that: The processing of the prediction generation module includes: The multi-scale modeling results are restored to time-domain seasonal forecast data through inverse discrete wavelet transform; Perform global Mamba modeling and global projection on the final global trend components in turn to obtain trend forecast data; The time domain seasonal forecast data and trend forecast data are linearly weighted fused, and the weighted fused data are output projected and denormalized in turn to obtain the predicted long-term time series.

7. A long-term multivariate time series prediction device based on multi-level decomposition and Mamba, characterized in that: include: Acquisition module, used to obtain the target multivariate time series; A prediction module, configured to input the target multivariate time series into a trained prediction model based on multi-level decomposition and Mamba to obtain a predicted long-term time series; The prediction model based on multi-level decomposition and Mamba consists of a pre-processing module, an adaptive time channel decomposition module, a mixed frequency and time domain multi-level decomposition module, a multi-scale Mamba modeling module and a prediction generation module connected in sequence.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 6 is implemented.

9. A computer program product, characterized in that The computer program product includes computer program code, and when the computer program code is run on a computer, the computer is enabled to implement the method according to any one of claims 1 to 6.

10. An electronic terminal comprising a memory, a processor, and a computer program stored in the memory, characterized in that: The processor executes the computer program to implement the method according to any one of claims 1 to 6.

Images