Training method and device of time sequence prediction model, equipment and storage medium

Abstract

The invention is applicable to the technical field of deep learning and data processing, and provides a time sequence prediction model training method, a time sequence prediction model comprises an input layer, a first branch network, a second branch network and an output layer, and the first branch network and the second branch network each comprise a MambaFFN module. The model training method comprises the steps of performing normalization processing on a training data set through an input layer, performing variable embedding processing on the normalized training data set through a first branch network to obtain a first representation, performing patch embedding processing on the normalized training data set through a second branch network to obtain a second representation, and performing patch embedding processing on the second representation to obtain a second representation. And performing fusion processing on the first representation and the second representation through an output layer to obtain a prediction result on each time step in the prediction time step, and performing iterative training on the time sequence prediction model by adopting a loss function based on the prediction results until the loss function converges to obtain the time sequence prediction model. Therefore, the training effect of the model and the precision and accuracy of the model on future prediction are improved.

Description

Technical Field

[0001] The present invention belongs to the field of deep learning and data processing technology, and in particular to a training method, device, equipment and storage medium for a time series prediction model. Background Art

[0002] Traditional statistical methods such as ETS (Exponential Smoothing) and ARIMA (AutoRegressive Integrated Moving Average), as well as machine learning techniques such as SVR (support vector regression) and GBRT (Gradient Boosting Decision Tree), although they perform well in some application scenarios, their prediction accuracy drops significantly when dealing with highly nonlinear and non-stationary time series. They rely on strong stationarity assumptions and are difficult to effectively capture complex time dependencies. In addition, these methods are weak in dealing with multivariate time series prediction tasks and are difficult to cope with the challenges of increasing data dimensions and complexity.

[0003] With the development of deep learning technology, complex neural network models such as Transformer are widely used in time series prediction. However, the computational complexity of Transformer models increases quadratically with the increase of input sequence length, resulting in huge consumption of computing resources when processing long sequences. This not only limits the practical feasibility of the model in large-scale applications, but also may affect the efficiency of real-time prediction tasks. In this regard, although some simpler linear models (such as DLinear and TiDE) can surpass complex Transformer models in some scenarios and show higher prediction efficiency, they still have shortcomings in processing complex time series data. These models are difficult to effectively capture the nonlinear characteristics and multivariate dependencies in time series, resulting in unsatisfactory prediction performance in complex application scenarios. Although the state space model (SSM) performs well in processing long sequence tasks, especially the Mamba model, which shows comparable performance to the Transformer in language modeling tasks and maintains linear time complexity, Mamba's global dependency capture capability in multivariate time series prediction is limited, resulting in poor performance when processing complex multivariate relationships. In addition, Mamba's unidirectional scanning mechanism limits its effectiveness in capturing multivariate dependencies and cannot fully utilize global information, which may lead to performance degradation in multivariate time series prediction tasks. Summary of the invention

[0004] The purpose of the present invention is to provide a training method, device, equipment and storage medium for a time series prediction model, aiming to solve the problem of poor prediction effect for multivariate time series prediction tasks caused by the prior art.

[0005] On the one hand, the present invention provides a training method for a time series prediction model, wherein the time series prediction model includes an input layer, a first branch network, a second branch network, and an output layer, wherein the first branch network and the second branch network both include a MambaFFN module, and the method includes the following steps:

[0006] Normalizing the received training data set through the input layer, wherein the training data set consists of multivariate time series data;

[0007] Performing variable embedding processing on the normalized training data set through the first branch network to obtain a first representation;

[0008] Performing patch embedding processing on the normalized training data set through the second branch network to obtain a second representation;

[0009] The first representation and the second representation are fused by the output layer to obtain a prediction result at each time step within a preset prediction time step;

[0010] Based on the prediction results, the time series prediction model is iteratively trained using a preset loss function until the loss function converges.

[0011] Preferably, the step of performing variable embedding processing on the normalized training data set through the first branch network to obtain a first representation includes:

[0012] Embedding the time series data of each variable in the training data set to obtain a corresponding variable embedding representation;

[0013] The variable embedding representations of all variables in the training data set are integrated through the MambaFFN module to obtain the first representation.

[0014] Preferably, the step of performing patch embedding processing on the normalized training data set through the second branch network to obtain a second representation includes:

[0015] Slicing the time series data of each variable in the training data set to obtain a plurality of time segments;

[0016] Embed each time segment to obtain the corresponding segment embedding representation;

[0017] All the fragment embedding representations corresponding to the same variable in the training data set are integrated through the MambaFFN module to obtain the second representation.

[0018] Preferably, based on the prediction result, the step of iteratively training the time series prediction model using a preset loss function until the loss function converges includes:

[0019] Calculate the error between the predicted result and the corresponding true value at each time step;

[0020] Performing exponential smoothing attenuation processing on the calculated errors corresponding to different time steps within the prediction time step to obtain a weighted error for each time step;

[0021] Based on the weighted error, a loss value of the loss function is obtained, and the model parameters of the time series prediction model are updated according to the loss value to minimize the loss of the loss function.

[0022]

[0023] Among them, ratio i =e -decay_factor·i , T is the prediction time step, is the prediction result generated at time step t+i based on the training data set A, X t+i (A) is the time step t+i, and The corresponding true value, ratio i is the weight of time step i, decay_factor is the preset decay factor, minE A is the expectation operation, which represents the minimization process on A.

[0024] Preferably, the MambaFFN module consists of two branches, one of which includes a linear layer and a sigmoid activation layer connected after the linear layer, and the other branch includes a linear layer, a feedforward neural network and a state-space model connected in sequence, wherein the feedforward neural network includes two linear layers, and a ReLU activation layer and a Dropout layer between the two linear layers.

[0025] On the other hand, the present invention provides a training device for a time series prediction model, the time series prediction model comprising an input layer, a first branch network, a second branch network and an output layer, wherein the first branch network and the second branch network both comprise a MambaFFN module, and the device comprises:

[0026] A data normalization unit, configured to perform normalization processing on the received training data set through the input layer, wherein the training data set is composed of multivariate time series data;

[0027] a variable embedding unit, configured to perform variable embedding processing on the normalized training data set through the first branch network to obtain a first representation;

[0028] a patch embedding unit, configured to perform patch embedding processing on the normalized training data set through the second branch network to obtain a second representation;

[0029] An information fusion unit, configured to fuse the first representation and the second representation through the output layer to obtain a prediction result at each time step within a preset prediction time step;

[0030] The model training unit is used to iteratively train the time series prediction model based on the prediction result using a preset loss function until the loss function converges.

[0031] Preferably, the variable embedding unit comprises:

[0032] An embedding representation obtaining unit, used to perform embedding processing on the time series data of each variable in the training data set to obtain a corresponding variable embedding representation;

[0033] The embedded representation integration unit is used to integrate the variable embedded representations of all variables in the training data set through the MambaFFN module to obtain the first representation.

[0034] On the other hand, the present invention also provides a computing device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the steps described in the above-mentioned method for training a time series prediction model are implemented.

[0035] On the other hand, the present invention also provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, it implements the steps described in the above-mentioned training method of a time series prediction model.

[0036] The time series prediction model of the present invention includes an input layer, a first branch network, a second branch network and an output layer, wherein the first branch network and the second branch network both include a MambaFFN module, and the training method of the model includes: normalizing the training data set through the input layer, performing variable embedding processing on the normalized training data set through the first branch network to obtain a first representation, performing patch embedding processing on the normalized training data set through the second branch network to obtain a second representation, fusing the first representation and the second representation through the output layer to obtain a prediction result at each time step within the prediction time step, and based on the prediction result, iteratively training the time series prediction model using a loss function until the loss function converges, thereby improving the training effect of the model and improving the precision and accuracy of the model for near-future predictions. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] Figure 1 is a schematic diagram of the structure of a time series prediction model provided in the first embodiment of the present invention;

[0038] Figure 2 It is a structural diagram of the MambaFFN module in the time series prediction model provided in the first embodiment of the present invention;

[0039] Figure 3 is a flow chart for implementing a training method for a time series prediction model provided in Embodiment 2 of the present invention;

[0040] Figure 4 is a flow chart for implementing a time series prediction method based on a time series prediction model obtained through training provided in Embodiment 3 of the present invention;

[0041] Figure 5 It is a structural schematic diagram of a training device for a time series prediction model provided in a fourth embodiment of the present invention;

[0042] Figure 6 It is a schematic diagram of the structure of the computing device provided in Embodiment 5 of the present invention. DETAILED DESCRIPTION

[0043] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0044] It should be understood that the term "plurality" in the present invention refers to two or more than two, and other quantifiers are similar.

[0045] In the present invention, the terms "first", "second", etc. are used to distinguish the same items or similar items with basically the same effects and functions. It should be understood that there is no logical or temporal dependency between "first", "second", and "nth", and there is no limitation on the quantity and execution order, unless otherwise specified.

[0046] The specific implementation of the present invention is described in detail below in conjunction with specific embodiments:

[0047] Embodiment 1:

[0048] Figure 1 FIG. 4 shows a schematic diagram of the structure of the time series prediction model provided by the first embodiment of the present invention. Figure 1 As shown, the time series prediction model applicable to the present invention includes an input layer, a first branch network, a second branch network and an output layer, wherein the first branch network and the second branch network both include a MambaFFN module.

[0049] In an embodiment of the present invention, a time series prediction model is used to predict future values ​​or trends based on historical time series data (referred to as historical information or historical sequence). Since a feedforward neural network (FFN) can perfectly retain global information and prevent overfitting, the causal convolution part used to enhance local relationship learning in the original Mamba is removed, and the FFN mechanism is introduced. The improved Mamba is called MambaFFN, thereby enhancing the ability of the MambaFFN module to learn long-distance dependencies, so that MambaFFN can more effectively capture remote dependencies, improve the accuracy of model prediction, and effectively prevent overfitting through the FFN mechanism, enhancing the generalization ability and robustness of the model.

[0050] In a possible embodiment, if Figure 2As shown, the MambaFFN module consists of two branches, one of which contains a linear layer (i.e., Linear layer) and a sigmod activation layer (i.e., σ) connected after the linear layer, and the other branch includes a linear layer, a feedforward neural network (i.e., FFN) and a state space model (i.e., SSM) connected in sequence, wherein the feedforward neural network contains two linear layers, and a ReLU (RectifiedLinearUnit, linear rectifier unit) activation layer and a Dropout layer (i.e., random inactivation layer) between the two linear layers, so that the MambaFFN module can enhance the time series prediction model's ability to capture complex nonlinear relationships while reducing the computational cost by strengthening the learning of nonlinear relationships, improve the generalization ability of the time series prediction model, and effectively prevent overfitting. Here, the specific calculation process of the MambaFFN module is expressed as follows:

[0051] h t =SSM(FFN(Linear(X t )))+σ(Linear(X t ))

[0052] FFN(X)=Linear2(Dropout(ReLU(Linear1(X))))).

[0053] Embodiment 2:

[0054] Figure 3 The implementation process of the training method of the time series prediction model provided by the second embodiment of the present invention based on the first embodiment is shown. For the convenience of explanation, only the part related to the embodiment of the present invention is shown, which is described in detail as follows:

[0055] In step S301, the received training data set is normalized through the input layer, wherein the training data set consists of multivariate time series data.

[0056] The embodiment of the present invention is applicable to computing devices, such as personal computers, servers, etc. In the embodiment of the present invention, the input layer of the time series prediction model is a normalization layer, which is used to perform normalization processing on the input time series data to obtain a normalized sequence. Here, a training data set consisting of multivariate time series data is input into the input layer of the time series prediction model, and the received training data set is normalized by the input layer. The multivariate time series data refers to time series data corresponding to multiple variables, and the time series data is a series of data points arranged in time order. These data points can be continuous or discrete, which reflects the state or observation value of the corresponding variable at different time points. In addition, the time series data of each variable can form a time series represented by a curve for the variable. For example, for the training data set of weather, it can include 8 variables such as air pressure, humidity, and rainfall. Each variable corresponds to a time curve, and the time curve reflects the time series data of the corresponding variable. For example, the time curve of air pressure reflects the specific air pressure value at different times.

[0057] In a feasible embodiment, the reversible instance normalization (Reversible Instance Normalization, RevIN) technology is used to normalize the time series data of each variable in the training data set to ensure that its mean is zero and the standard deviation is one, thereby alleviating the distribution shift problem in time series prediction and enhancing the prediction accuracy of subsequent models.

[0058] In step S302, variable embedding processing is performed on the normalized training data set through a first branch network to obtain a first representation.

[0059] In an embodiment of the present invention, the first branch network of the time series prediction model is also called a variable embedding (Variate-Embedding) branch network, which is used to perform variable embedding processing on the time series data corresponding to each variable output by the normalization layer to obtain an embedded multi-variable association representation, which is the first representation.

[0060] In a feasible embodiment, the generation of the first representation is achieved through the following steps:

[0061] (1) Embed the time series data of each variable in the training data set to obtain the corresponding variable embedding representation;

[0062] In the embodiment of the present invention, in order to capture the global features of each variable and explore the correlation between multiple variables, the time series data of each variable in the training data set is embedded. Specifically, the entire time series data of each variable is independently mapped into a high-dimensional variable embedding representation (variable token for short) to capture the global features of a single variable. This embedding process can be expressed as: variate =VariateEmbedding(X var ), which enables the model to establish more detailed interactions between multiple variables and improve the accuracy of prediction. var is the time series data of the normalized variable var, X variate For X var The variable token obtained after embedding processing.

[0063] In a feasible embodiment, the first branch network includes not only the MambaFFN module, but also a variable embedding module, through which the time series data of each variable in the training data set is embedded to obtain a corresponding variable embedding representation.

[0064] (2) The variable embedding representations of all variables in the training dataset are integrated through the MambaFFN module to obtain the first representation.

[0065] In the embodiment of the present invention, the variable token of each variable in the training data set is input into the MambaFFN module, and the variable tokens of different variables are integrated through the MambaFFN module to generate a multivariate association representation X v , this process can be expressed as: v =MambaFFN(X variate), here, since there is a lack of definite causal relationship between different variables, the causal convolution in the original Mamba block may affect the model performance. In addition, some studies believe that the causal convolution of the look-back window is unnecessary in time series analysis, because this may limit the extraction of temporal features, which also indicates that the causal convolution in the original Mamba block is unnecessary. At the same time, since the original Mamba block may forget global information, the use of the MambaFFN module in the first branch network can effectively prevent possible global information loss, thereby capturing the complex dependencies between different variables and enhancing the model's ability to learn multivariate associations between variables. That is, for the MambaFFN module, the number of variables in the training data set is the number of variables to learn. For example, a weather data set has 8 variables, including air pressure, humidity, rainfall, etc., then it is possible to learn the dependencies between any of these 8 variables, such as air pressure and humidity, air pressure and rainfall, humidity and rainfall, etc., to enhance the retention of global information.

[0066] The generation of the first representation is achieved through the above steps (1) and (2). Compared with the multi-variable embedding method of the traditional Transformer model, the first branch network significantly enhances the model's ability to capture the interaction relationship between variables by independently mapping the variable sequence, thereby improving the prediction accuracy.

[0067] In step S303, the normalized training data set is subjected to patch embedding processing through a second branch network to obtain a second representation.

[0068] In an embodiment of the present invention, the second branch network of the time series prediction model is also called a patch embedding branch network, which is used to divide the time series data corresponding to each variable output by the normalization layer into multiple patches of fixed length and perform embedding processing, so as to learn each variable separately in the time dimension, and obtain the time series dependency representation of each variable, which is the second representation.

[0069] In a feasible embodiment, the generation of the second representation is achieved through the following steps:

[0070] ① Slice the time series data of each variable in the training data set to obtain several time segments;

[0071] In the embodiment of the present invention, the time series data of each variable in the training data set is divided into multiple overlapping or non-overlapping segments according to the time step to retain the local information of the sequence, and a plurality of patches X of fixed length are obtained. patch(i.e., time slices), specifically, the time series data of each variable in the training data set is sliced ​​according to the preset slice time step and step time step, where the slice time step is the slice length of each patch and the step time step is the interval between patches. As an example, if the slice time step is 16 and the step time step is 8, it means that the slice length of each patch after slicing is 16 time steps, and the interval between patches is 8 time steps, that is, the first slice is 0-15, the second slice is 7-22, the third slice is 14-29... and so on.

[0072] In a feasible embodiment, the time series data of each variable is divided into a plurality of patches of fixed length by a time series slicing module in the second branch network, thereby retaining the local information of the sequence.

[0073] ②Embed each time segment to obtain the corresponding segment embedding representation;

[0074] In the embodiment of the present invention, each time segment is embedded to obtain a corresponding segment embedding representation. Specifically, each time segment X patch Mapped into a fragment embedding representation X d (short for fragment token) to capture the local information in the sequence. This mapping process can be expressed as: d =PatchEmbedding(X patch ), thereby enhancing the model’s ability to represent local information through sharding and fragment mapping, enabling the model to more accurately learn and capture short-term local features in time series while reducing computation and memory usage.

[0075] In a feasible embodiment, each patch is mapped to a fragment token through a patch embedding module in the second branch network, thereby effectively reducing computational and memory overheads.

[0076] ③ All fragment embedding representations corresponding to the same variable in the training data set are integrated through the MambaFFN module to obtain the second representation.

[0077] In the embodiment of the present invention, after all the fragment tokens of the same variable are input into the MambaFFN module, the different fragment tokens of the variable are integrated by the MambaFFN module to learn the time dependency relationship within the variable and generate the time series dependency representation X of the variable. p (i.e., the second representation), this integration process can be expressed as: This ensures the model's ability to capture the time dependence of the same variable, where are all different fragment tokens of the same variable in the training dataset, and n is the total number of time fragments that the variable is fragmented into. In addition, in this branch, the MambaFFN module has a better prediction effect than the original Mamba block. This is because the time series data of each variable has been divided into multiple patches, and the local information of each variable has been retained, so there is no need to enhance the local dependency through the causal convolution in the original Mamba block. Considering that the original Mamba block also tends to give priority to proximal information, FFN that can retain global information is very necessary. Therefore, in the second branch network, it is very natural and appropriate to use the MambaFFN module to further process all the fragment embedding representations corresponding to the same variable in the training dataset, which effectively avoids the information being limited to the local range.

[0078] The generation of the second representation is achieved through the above steps ① to ③. It can be seen that the second branch network retains local information through sharding and uses the MambaFFN module to enhance the learning of global information, which is more suitable for time series prediction of long sequences.

[0079] In step S304, the first representation and the second representation are fused through the output layer to obtain a prediction result at each time step within a preset prediction time step.

[0080] In the embodiment of the present invention, when the first representation and the second representation are fused through the output layer, specifically, the X from the first branch network is first fused through the output layer. v With X from the second branch network p Directly add, then perform denormalization to generate the final prediction result X at each time step within the preset prediction time step pred The entire fusion processing operation can be expressed as:

[0081] X all =X v +X p

[0082] X pred =RevIN -1 (X all ),

[0083] Thus, the prediction performance of the model is improved by combining the dependency between variables and time dependency.

[0084] In step S305, based on the prediction results, the time series prediction model is iteratively trained using a preset loss function until the loss function converges.

[0085] In the embodiment of the present invention, in order to better adapt to the characteristics of time series prediction, an exponentially weighted signal decay loss function (Exponentially Weighted Signal Decay Loss, EWSDL) is introduced as the loss function of the time series prediction model. The loss function applies an exponential decay factor to the error weight, making the loss function more sensitive to the near future prediction error, thereby improving the accuracy of the model for near future prediction. Here, based on the prediction results, the time series prediction model is iteratively trained using EWSDL until the loss function converges.

[0086] In a feasible embodiment, based on the prediction results, the time series prediction model is iteratively trained using EWSDL until the loss function converges. The error between the prediction result at each time step and the corresponding true value is first calculated, and then the calculated errors corresponding to different time steps within the prediction time step are exponentially smoothed and decayed to obtain the weighted error of each time step. Finally, based on the weighted error, the loss value of the loss function is obtained, and the model parameters of the time series prediction model are updated according to the loss value. The above steps S301 to S305 are repeated until the loss of the loss function is minimized. By weighting the errors of the time steps, the errors in the near future have a greater impact on the total loss, while the errors in the far future have a lower weight, thereby enhancing the sensitivity of near future predictions.

[0087] In yet another possible embodiment, the EWSDL is Among them, ratio i =e -decay_factor·i ;

[0088] T is the prediction time step, that is, the number of future moments predicted by the model, which determines the number of time steps to be considered in the loss function, that is, the total number of steps predicted by the model;

[0089] t is the prediction start time in time series prediction. It can also be understood that t is the last time point of the input historical sequence. Next, it is necessary to predict the values ​​of the next T time steps based on these historical sequences, that is, t+1 to t+T. As an example, when training a time series prediction model, a sliding window method is usually used to divide the historical sequence. Under this method, the last time point in the window is t. Assuming that the length of the window is W, the time series data contained therein is [t-W+1, t]. This t is used to predict the starting time point of the next T steps.

[0090] i represents the predicted time step, from 1 to T, which is used to traverse all prediction time steps to ensure that the loss function covers the entire prediction window;

[0091] ratio iis the weight of time step i, which is an exponentially decaying factor used to weight the errors of different time steps, so that the weight of the prediction error in the near future (smaller i value) is higher, while the weight of the prediction error in the far future (larger i value) is lower. In this way, the model will pay more attention to optimizing the prediction accuracy in the near future;

[0092] decay_factor is the preset decay factor to control the speed of exponential decay, which determines the decay speed of the weight. A larger decay factor will make the weight decay faster, and the error in the near future will be more important. A smaller decay factor will make the weight decay slower, and the error in the far future will have a greater impact. Specifically, set decay_factor to 0.005;

[0093] It is the normalization factor of the loss function, which is used to calculate the average loss over the time step to ensure that the calculation of the loss function is not affected by the different number of prediction steps and to keep the loss comparable between different models;

[0094] is the prediction result generated at time step t+i based on historical information A (i.e., past time series data, which can be a training data set for model training here), as the output of the model, used to compare with the true value to calculate the prediction error;

[0095] X t+i (A) is the time step t+i, and The corresponding true value;

[0096] minE A is the expected operation, which represents the minimization process on A, that is, by minimizing the loss function, the model performs best on a given A.

[0097] Therefore, through this loss function, a higher weight is given to the error of near-future prediction, so that the model can more accurately capture the trend changes in the near future and improve the accuracy of short-term predictions. At the same time, an exponential decay method is adopted for the error of far-future prediction, which reduces the impact of these high-variance predictions on the overall loss, reduces the negative impact of far-future predictions, and increases the stability of the model. In addition, the exponential decay factor in EWSDL can be flexibly adjusted according to task requirements to ensure that the loss function can adapt to the characteristics of different data sets and better serve the practical application of time series prediction.

[0098] In an embodiment of the present invention, a time series prediction model includes an input layer, a first branch network, a second branch network and an output layer, wherein the first branch network and the second branch network both include a MambaFFN module, thereby realizing a dual-branch embedding architecture including a variable embedding branch and a patch embedding branch, which can simultaneously capture the time dependency and the relationship between variables in the time series, and improve the modeling ability of complex time series data. When training the time series prediction model, the training data set is first normalized through the input layer, and then the normalized training data set is subjected to variable embedding processing through the first branch network to obtain a first representation, and the normalized training data set is subjected to patch embedding processing through the second branch network to obtain a second representation. After that, the first representation and the second representation are fused through the output layer to obtain a prediction result at each time step within the prediction time step. Finally, based on the prediction result, the time series prediction model is iteratively trained using a loss function until the loss function converges, thereby achieving a more comprehensive understanding and processing of the complexity of the time series data by simultaneously paying attention to the relationship between time and variables, and improving the training effect of the model and the precision and accuracy of the model for near-future predictions.

[0099] Embodiment three:

[0100] Figure 4 The implementation process of the time series prediction method based on the time series prediction model trained in the second embodiment provided by the third embodiment of the present invention is shown. For the convenience of explanation, only the part related to the embodiment of the present invention is shown, which is described in detail as follows:

[0101] The target data set to be predicted is input into the time series prediction model obtained by the training method of Example 2 to predict the near future situation of each variable in the target data set. Specifically, the time series prediction is achieved through the following steps:

[0102] In step S401, the received target data set is normalized through the input layer, wherein the target data set is composed of multivariate target time series data.

[0103] In step S402, variable embedding processing is performed on the normalized target data set through a first branch network to obtain a first representation.

[0104] In step S403, the normalized target data set is subjected to patch embedding processing through a second branch network to obtain a second representation.

[0105] In step S404, the first representation and the second representation are fused through the output layer to obtain a prediction result at each time step within a preset target prediction time step.

[0106] In the embodiment of the present invention, since this embodiment is the application process of the time series prediction model, and the principle of the model application process is similar to the principle of the training process, the specific implementation of steps S401-step S404 can refer to the description of steps S301-S304 of embodiment 2, and will not be repeated here.

[0107] Embodiment 4:

[0108] Figure 5 The structure of the training device of the time series prediction model based on the first embodiment provided by the fourth embodiment of the present invention is shown. For the convenience of description, only the part related to the embodiment of the present invention is shown, including:

[0109] A data normalization unit 51 is used to perform normalization processing on the received training data set through the input layer, wherein the training data set is composed of multivariate time series data;

[0110] A variable embedding unit 52, configured to perform variable embedding processing on the normalized training data set through a first branch network to obtain a first representation;

[0111] a patch embedding unit 53, configured to perform patch embedding processing on the normalized training data set through a second branch network to obtain a second representation;

[0112] An information fusion unit 54 is used to fuse the first representation and the second representation through an output layer to obtain a prediction result at each time step within a preset prediction time step;

[0113] The model training unit 55 is used to iteratively train the time series prediction model based on the prediction results using a preset loss function until the loss function converges.

[0114] Wherein, preferably, the variable embedding unit 52 includes:

[0115] The embedding representation obtaining unit is used to embed the time series data of each variable in the training data set to obtain the corresponding variable embedding representation;

[0116] The embedding representation integration unit is used to integrate the variable embedding representations of all variables in the training data set through the MambaFFN module to obtain a first representation.

[0117] Preferably, the patch embedding unit 53 comprises:

[0118] The sequence slicing unit is used to slice the time series data of each variable in the training data set into several time segments;

[0119] A fragment embedding unit is used to embed each time fragment to obtain a corresponding fragment embedding representation;

[0120] The fragment integration unit is used to integrate all the fragment embedding representations corresponding to the same variable in the training data set through the MambaFFN module to obtain a second representation.

[0121] Preferably, the model training unit 55 includes:

[0122] The error calculation unit is used to calculate the error between the prediction result and the corresponding true value at each time step;

[0123] The error smoothing unit is used to perform exponential smoothing attenuation processing on the calculated errors corresponding to different time steps within the prediction time step to obtain the weighted error of each time step;

[0124] The parameter updating unit is used to obtain the loss value of the loss function based on the weighted error, and update the model parameters of the time series prediction model according to the loss value to minimize the loss of the loss function.

[0125] Preferably, the loss function is Among them, ratio i =e -decay_factor·i , T is the prediction time step, is the prediction result generated at time step t+i based on the training dataset A, X t+i (A) is the time step t+i, and The corresponding true value, ratio i is the weight of time step i, decay_factor is the preset decay factor, minE A is the expectation operation, which represents the minimization process on A.

[0126] Preferably, the MambaFFN module consists of two branches, one of which includes a linear layer and a sigmoid activation layer connected after the linear layer, and the other branch includes a linear layer, a feedforward neural network and a state-space model connected in sequence, wherein the feedforward neural network includes two linear layers, and a ReLU activation layer and a Dropout layer between the two linear layers.

[0127] In the embodiment of the present invention, each unit of the training device of the time series prediction model can be implemented by a corresponding hardware or software unit, and each unit can be an independent software or hardware unit, or can be integrated into a software or hardware unit, which is not intended to limit the present invention. Specifically, the implementation of each unit can refer to the description of the aforementioned embodiments 1 and 2, and will not be repeated here.

[0128] Embodiment five:

[0129] Figure 6The structure of a computing device provided in Embodiment 5 of the present invention is shown. For the sake of convenience, only the parts related to the embodiment of the present invention are shown.

[0130] The computing device 6 of the embodiment of the present invention includes a processor 60, a memory 61, and a computer program 62 stored in the memory 61 and executable on the processor 60. When the processor 60 executes the computer program 62, the steps in the training method embodiment of the above-mentioned time series prediction model are implemented, for example Figure 3 Alternatively, when the processor 60 executes the computer program 62, the functions of each unit in the above-mentioned device embodiments are realized, for example Figure 5 Function of the unit shown.

[0131] In an embodiment of the present invention, a time series prediction model includes an input layer, a first branch network, a second branch network and an output layer, wherein the first branch network and the second branch network both include a MambaFFN module, and a training method of the model includes: normalizing a training data set through the input layer, performing variable embedding processing on the normalized training data set through the first branch network to obtain a first representation, performing patch embedding processing on the normalized training data set through the second branch network to obtain a second representation, fusing the first representation and the second representation through the output layer to obtain a prediction result at each time step within the prediction time step, and based on the prediction result, iteratively training the time series prediction model using a loss function until the loss function converges, thereby improving the training effect of the model and improving the precision and accuracy of the model for near-future predictions.

[0132] The computing device of the embodiment of the present invention may be a personal computer or a server. The steps implemented when the processor 60 in the computing device 6 executes the computer program 62 to implement a training method for a time series prediction model can refer to the description of the aforementioned method embodiment, which will not be repeated here.

[0133] Embodiment six:

[0134] In an embodiment of the present invention, a computer-readable storage medium is provided, which stores a computer program. When the computer program is executed by a processor, the steps in the above-mentioned training method embodiment of a time series prediction model are implemented, for example, Figure 3 Alternatively, when the computer program is executed by a processor, the functions of each unit in the above-mentioned device embodiments are realized, for example Figure 5 Function of the unit shown.

[0135] In an embodiment of the present invention, a time series prediction model includes an input layer, a first branch network, a second branch network and an output layer, wherein the first branch network and the second branch network both include a MambaFFN module, and a training method of the model includes: normalizing a training data set through the input layer, performing variable embedding processing on the normalized training data set through the first branch network to obtain a first representation, performing patch embedding processing on the normalized training data set through the second branch network to obtain a second representation, fusing the first representation and the second representation through the output layer to obtain a prediction result at each time step within the prediction time step, and based on the prediction result, iteratively training the time series prediction model using a loss function until the loss function converges, thereby improving the training effect of the model and improving the precision and accuracy of the model for near-future predictions.

[0136] The computer-readable storage medium of the embodiment of the present invention may include any entity or device or recording medium capable of carrying computer program code, for example, ROM / RAM, magnetic disk, optical disk, flash memory and other memories.

[0137] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A training method for a time series prediction model, characterized in that: The time series prediction model includes an input layer, a first branch network, a second branch network and an output layer, wherein the first branch network and the second branch network both include a MambaFFN module, and the method includes the following steps: Normalizing the received training data set through the input layer, wherein the training data set consists of multivariate time series data; Performing variable embedding processing on the normalized training data set through the first branch network to obtain a first representation; Performing patch embedding processing on the normalized training data set through the second branch network to obtain a second representation; The first representation and the second representation are fused by the output layer to obtain a prediction result at each time step within a preset prediction time step; Based on the prediction results, the time series prediction model is iteratively trained using a preset loss function until the loss function converges.

2. The method according to claim 1, characterized in that The step of performing variable embedding processing on the normalized training data set through the first branch network to obtain a first representation includes: Embedding the time series data of each variable in the training data set to obtain a corresponding variable embedding representation; The variable embedding representations of all variables in the training data set are integrated through the MambaFFN module to obtain the first representation.

3. The method according to claim 1, characterized in that The step of performing patch embedding processing on the normalized training data set through the second branch network to obtain a second representation includes: Slicing the time series data of each variable in the training data set to obtain a plurality of time segments; Embed each time segment to obtain the corresponding segment embedding representation; All the fragment embedding representations corresponding to the same variable in the training data set are integrated through the MambaFFN module to obtain the second representation.

4. The method according to claim 1, characterized in that Based on the prediction result, the step of iteratively training the time series prediction model using a preset loss function until the loss function converges includes: Calculate the error between the predicted result and the corresponding true value at each time step; Performing exponential smoothing attenuation processing on the calculated errors corresponding to different time steps within the prediction time step to obtain a weighted error for each time step; Based on the weighted error, a loss value of the loss function is obtained, and the model parameters of the time series prediction model are updated according to the loss value to minimize the loss of the loss function.

5. The method according to claim 4, characterized in that The loss function is Among them, ratio i =e -decay_factor·i , T is the prediction time step, t is the starting time of the prediction, is the prediction result generated at time step t+i based on historical information A, X t+i (A) is the time step t+i, and The corresponding true value, ratio i is the weight of time step i, decay_factor is the preset decay factor, minE A is the expectation operation, which represents the minimization process on A.

6. The method according to claim 1, characterized in that The MambaFFN module consists of two branches, one of which includes a linear layer and a sigmoid activation layer connected after the linear layer, and the other branch includes a linear layer, a feedforward neural network and a state-space model connected in sequence, wherein the feedforward neural network includes two linear layers, and a ReLU activation layer and a Dropout layer between the two linear layers.

7. A training device for a time series prediction model, characterized in that: The time series prediction model includes an input layer, a first branch network, a second branch network and an output layer, wherein the first branch network and the second branch network both include a MambaFFN module, and the device includes: A data normalization unit, configured to perform normalization processing on the received training data set through the input layer, wherein the training data set is composed of multivariate time series data; a variable embedding unit, configured to perform variable embedding processing on the normalized training data set through the first branch network to obtain a first representation; a patch embedding unit, configured to perform patch embedding processing on the normalized training data set through the second branch network to obtain a second representation; An information fusion unit, configured to fuse the first representation and the second representation through the output layer to obtain a prediction result at each time step within a preset prediction time step; The model training unit is used to iteratively train the time series prediction model based on the prediction result using a preset loss function until the loss function converges.

8. The device according to claim 7, characterized in that The variable embedding unit comprises: An embedding representation obtaining unit, used to perform embedding processing on the time series data of each variable in the training data set to obtain a corresponding variable embedding representation; The embedded representation integration unit is used to integrate the variable embedded representations of all variables in the training data set through the MambaFFN module to obtain the first representation.

9. A computing device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images