Time sequence probability prediction method and system based on Coupman theory and Kalman filtering and application of time sequence probability prediction method and system based on Coupman theory and Kalman filtering

Abstract

The invention discloses a time sequence probability prediction method based on the Kumpmann theory and Kalman filtering, and the method comprises the following steps: 1, carrying out the patching of input time sequence data, and mapping the data to a high-dimensional potential space; 2, mapping the time sequence data after patch mapping to a high-dimensional measurement space, and modeling a linear dynamic system through a Kupman operator; step 3, optimizing the linear dynamic system by using a Kalman network, modeling the state of the linear dynamic system, updating state estimation, and outputting potential variables; and step 4, carrying out resampling from variational posteriori distribution, mapping the potential variables back to an original space, carrying out modeling to predict target distribution, and outputting a probabilistic prediction result of a time sequence. The invention further discloses a prediction system for realizing the prediction method, and the prediction system has a wide application scene.

Description

Technical Field

[0001] The present invention belongs to the technical field of time series prediction, and relates to a time series probability prediction method, system and application based on Koopman theory and Kalman filtering. Background Art

[0002] Time series forecasting is a key task in machine learning, widely used in fields such as finance, meteorology, and transportation. Probabilistic forecasting methods have garnered significant attention in recent years. Unlike traditional point-by-point forecasting methods, probabilistic forecasting not only provides a predicted value but also provides information about the uncertainty of the forecast. By outputting a predictive distribution, probabilistic forecasting can help decision makers understand and address uncertainty in the forecast. This confidence information is particularly important in high-risk fields such as finance and healthcare. Probabilistic forecasting models, such as deep generative models based on Monte Carlo simulation and variational inference methods, have become a research hotspot in time series forecasting. The variational autoencoder (VAE), as a generative model, introduces latent variables and variational inference to model the generative process of time series data, thereby enabling probabilistic predictions of future time steps.

[0003] Although deep learning methods have performed well in short-term prediction of time series, existing models still have significant error accumulation problems in long-term prediction tasks. In order to alleviate this problem, some studies in recent years have [1-3] We are beginning to explore ways to enhance the robustness of deep learning models by incorporating physical models. For example, Koopman theory provides an effective modeling approach by transforming nonlinear systems into approximate linear systems. The Koopman operator can approximate complex nonlinear systems and achieve predictions using existing linear dynamic systems theory, providing new perspectives and tools for deep learning models.

[0004] At the same time, the Kalman filter, a classic state estimation method for linear dynamic systems, has been widely used in time series forecasting. By recursively updating the covariance matrix of the forecast error, the Kalman filter effectively models system uncertainty and reduces error accumulation in long-term forecasts. Consequently, the combination of the Kalman filter and deep learning models has become a research trend in recent years. Several studies have attempted to combine deep neural networks with the Kalman filter to achieve accurate modeling of time series dynamics and effectively address the error problem in long-term forecasts.

[0005] While existing research has provided a variety of solutions for probabilistic time series forecasting, balancing model complexity and forecast accuracy remains a challenge. Existing deep learning-based time series forecasting methods often focus on a single prediction objective, ignoring the uncertainty in the system. While linearization methods based on physical models can effectively simplify the problem, they often fail to capture the richness of the data when dealing with high-dimensional, nonlinear time series. Summary of the Invention

[0006] In order to address the deficiencies in the prior art, the present invention aims to provide a time series probability prediction method, system and application based on Koopman theory and Kalman filtering.

[0007] The time series probability prediction method in this invention combines the Koopman theory and the Kalman filtering method, which can transform nonlinear time series into linear dynamic systems, and refine the prediction results through the Kalman network, reducing error accumulation and improving the accuracy of long-term predictions.

[0008] The present invention provides a time series probability prediction method based on Koopman theory and Kalman filtering, the prediction method comprising the following steps:

[0009] Step 1: Patch the input time series data and map it into a high-dimensional latent space;

[0010] Step 2: Map the patched time series data into a high-dimensional measurement space and model the linear dynamic system using the Koopman operator;

[0011] Step 3: Optimize the linear dynamic system using a Kalman network, model the state of the linear dynamic system, update the state estimate, and output potential variables;

[0012] Step 4: Resample from the variational posterior distribution, map the latent variables back to the original space, model and predict the target distribution, and output the probabilistic prediction results of the time series.

[0013] In step 1, the input time series data is nonlinear; patching refers to dividing the input time series into equal-interval divisions to form multiple time series sub-segments of equal length, same interval and non-overlapping as patches, giving the basic unit of modeling stronger semantics.

[0014] Specifically, the input time series data is patched Divide into one or more non-overlapping patches

[0015] Where s represents the patch size, n represents the number of patches, represents the i-th patch;

[0016] X P Mapped to a high-dimensional latent space, denoted as X P′ =Projection(Flatten(X P ));

[0017] Specifically, first patch X P Flattened to Mapping to a high-dimensional embedding space through linear projection

[0018] In step 2, the patched time series data after input is mapped into a high-dimensional measurement space by using a learnable multilayer perceptron network as the measurement function, which is expressed as Perform linearization processing; where ψ is the measurement function, Markers representing projections into the measurement space;

[0019] The Koopman operator for constructing the linear dynamic system includes the local Koopman operator and the global Koopman operator The complete Koopman operator is expressed as

[0020] In the present invention, for the nonlinear system x k+1 =f(x k ), where x k Represents the system state, f is a nonlinear function, and it is assumed that the state of the system can be mapped into the measurement function space ψ. The linear dynamic system is modeled by the infinite-dimensional linear Koopman operator in the measurement space, which is expressed as follows:

[0021]

[0022] Wherein, the local Koopman operator The single-step extended dynamic mode decomposition technique is used to fit the following:

[0023]

[0024] in, and In the selection of high-dimensional representation, a single-step dislocation method is adopted. express The MP pseudo-inverse, Can be and The product representation of ,approximately describes the linear transfer process of high-dimensional representation;

[0025] By introducing a learnable global Koopman operator Alleviate the convergence anomaly problem in the measurement space due to insufficient fitting of the measurement function and capture the globally shared dynamics;

[0026] The linear dynamic system constructed by the complete Koopman operator is expressed as follows:

[0027]

[0028] Among them, the first high-dimensional representation and Koopman operator to describe all high-dimensional representations on the look-back window and the prediction window and

[0029] The linear dynamic system is a biased linear dynamic system.

[0030] The Koopman network outputs residual information and a high-dimensional representation of the prediction window. The residual information is expressed as follows:

[0031] is a high-dimensional representation on the look-back window.

[0032] In step 3, the nonlinear residual information is reused in the Kalman network and integrated into the linear dynamic system constructed by the Koopman network to optimize the linear dynamic system and improve the accuracy of the linear system; the matrices in the Kalman network can all be learned;

[0033] Specifically, the residual information is reused and integrated through the integrator in the Kalman network, which is expressed as follows:

[0034]

[0035] U=Integrator(Residual)=[u1,u2,…,u m ]

[0036] Among them, Residual is the residual information output by the Koopman network, which is represented by the actual P* Representation of reconstruction with the Koopman operator The difference is obtained, and U is the further integration result of the residual information, which is integrated through a Transformer-based integrator.

[0037] Then the state transition of the system is described by the process model, which is expressed as follows:

[0038] z k =Az k-1 +Bu k +w k

[0039]

[0040] in, is the state transition matrix, is the control input matrix, is the process noise, Q is the covariance matrix of the process noise; u k Belongs to the aforementioned integrated residual information U; z k It is the result of step k of the transfer process, and the initial state z0 is assigned to The last high-dimensional representation in

[0041] By observing the model k =Hz k +v k Update the system status, including: is the observation matrix, and the predicted value is considered as a prior observation in the update step, is the observation noise, R is the covariance matrix of the observation noise;

[0042] In a specific embodiment, the covariance matrix of the process noise and the covariance matrix of the observation noise are initialized to the identity matrix and the positive definiteness is maintained using a lower triangular matrix, which is expressed as follows: and

[0043] Then, the prediction step and update step are iterated, and the prediction step is expressed as:

[0044]

[0045] in, is the state transition matrix, is the control input matrix, is the state predicted at step k during the iteration, is the process uncertainty covariance matrix predicted in the kth step during the iteration; u k Belongs to the aforementioned integrated residual information U, z k-1 and P k-1 is the state in the k-1th step and the process uncertainty covariance matrix, and Q is the covariance matrix of the process noise.

[0046] The update step is done by the Kalman gain K k Measure the weight between observations and predictions, and update the linear system:

[0047]

[0048] Among them, z kand P k It is based on the Kalman gain K k The updated state vector and the covariance matrix of the process uncertainty; Kalman gain K k The process uncertainty covariance matrix predicted by the k-th step in the iterative process The observation matrix H and the covariance matrix R of the observation noise are calculated collaboratively.

[0049] Then, the updated state vector Z = [z1, z2, ..., z m ] and the covariance matrix of process uncertainty P = [P1, P2, ..., P m ], which describe the uncertainty of the time process in a dynamic system. In order to fully utilize the power of the integrator, a residual connection is introduced: Z′=Z+U, where Z′ is the predicted latent variable and U is the integration result of the residual information.

[0050] After obtaining the predicted latent variable Z′ and the covariance matrix P of the process uncertainty, the posterior distribution is varied Resample and map the resampled latent variables back to the original space through Gaussian distribution To model the target distribution of the prediction, μ and σ represent the mean and standard deviation respectively. In this way, the distribution on the prediction target window Y is obtained, and samples of Y can be obtained from it to complete the probability prediction.

[0051] The present invention also proposes a prediction system for implementing the above-mentioned time series probability prediction method, wherein the prediction system comprises: a data preprocessing module, a Koopman network module, a Kalman network module, a decoder module, and a prediction result output module;

[0052] The data preprocessing module is used for input labeling and embedding. It receives time series data and performs segmentation processing on it, divides the time series data into multiple segments and maps them into a high-dimensional latent space to extract feature representations of the time series;

[0053] The Koopman network module is used to map time series data to a high-dimensional measurement space based on the Koopman theory, and to construct a linear dynamic system using the Koopman operator to achieve linear modeling of the time series;

[0054] The Kalman network module is used to optimize the linear dynamic system generated by the Koopman network based on the Kalman filter, model the state of the linear dynamic system, update the state estimate, and output the potential variables;

[0055] The decoder module is used to resample from the variational posterior distribution, map the latent variables back to the original space, model and predict the target distribution, and obtain a probabilistic prediction result of the time series;

[0056] The prediction result output module is used to output the probability prediction result of the time series.

[0057] The present invention also provides applications of the above-mentioned prediction method or the above-mentioned prediction system in time series probability prediction, including short-step probability prediction scenarios and long-step probability prediction scenarios.

[0058] The beneficial effects of the present invention include: compared with the traditional time series prediction method, the K 2 The VAE model can achieve good results in both short-term and long-term prediction scenarios, especially in dealing with the error accumulation problem in long-term predictions and effectively modeling the uncertainty in time series. By combining the linearization capability of the Koopman theory with the uncertainty processing of the Kalman filter, the K 2 The VAE model can provide efficient probabilistic predictions and provide decision makers with more comprehensive prediction information; in addition, it has excellent lightweight effects. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without paying any creative work.

[0060] Figure 1 K of the present invention 2 The data flow process of VAE.

[0061] Figure 2 K of the present invention 2 Schematic diagram of the VAE architecture.

[0062] Figure 3 The figure is a comparison chart of the effects of the method of the present invention compared with the prior art in a short-step prediction scenario.

[0063] Figure 4 This is a comparison chart of the effects of the method of the present invention compared with the prior art in a long-step prediction scenario.

[0064] Figure 5 The figure is a comparison between the lightweight effect of the method of the present invention and the prior art. DETAILED DESCRIPTION

[0065] The present invention is further described in detail with reference to the following specific examples and accompanying drawings. The processes, conditions, experimental methods, etc. for implementing the present invention, except for those specifically mentioned below, are common knowledge and common common sense in the art and are not particularly limited by the present invention.

[0066] This invention provides a method for probabilistic time series prediction based on the Koopman theory and Kalman filtering. This method, based on a variational autoencoder using the Koopman theory and Kalman filtering, operates through the following steps: first, block coding is used to enhance the semantics of time series encoding. A Koopman network is then used to convert the nonlinear time series into a linear dynamic system. This linear dynamic system is then optimized using a Kalman network to reduce the accumulation of prediction errors and model uncertainty in the latent space. Finally, a decoder is used to model the target distribution in the original space to achieve probabilistic prediction. This method can significantly improve the accuracy of long-term predictions, overcoming the shortcomings of existing methods in this area. The invention demonstrates superior performance in both short-term and long-term probabilistic time series prediction, providing a more efficient and accurate prediction solution.

[0067] For the input window of the time series The ultimate goal of the time series probability prediction method in this invention is to Constructing the target distribution. Considering the time series fragments, i.e. “patches” as basic elements can retain most semantic information and help establish meaningful state transition processes for autoregressive models. The K-CNN composed of the input sequence embedding module, Koopman network, Kalman network and decoder in the present invention is 2 VAE is also similar to an autoregressive dynamic system, which models the state transition process through the hidden space. 2 VAE uses multivariate segments as basic units, implicitly modeling the interaction between variables during state transition. Consider the initial nonlinear input time series Divide the nonlinear input time series into multiple non-overlapping patches:

[0068]

[0069] Where s represents the patch size, n represents the number of patches, represents the i-th patch. Then, X P Mapped to a high-dimensional latent space: X P′ =Projection(Flatten(X P )); During the mapping process, patch X P First it is flattened into Then it is mapped to The high-dimensional embedding space of

[15] is used to fuse variable information for nonlinear time series modeling problems in the real world due to linearization.

[0070] After the patches are mapped to the high-dimensional latent space, they can be used to further extract the latent features of the time series.

[0071] K2 VAE uses the Koopman theory to construct a measurement function, projecting the system state represented by the patched time series data into a measurement space that can be modeled by a linear system, transforming it into a linear dynamic system in the measurement function space. The present invention uses a learnable multilayer perceptron network as the measurement function ψ: in Represents a marker projected into the measurement space.

[0072] To capture the local transition rules, a single-step extended dynamic mode decomposition technique is used to efficiently fit the optimal local Koopman operator.

[0073]

[0074]

[0075] in, express The MP pseudo-inverse, and The single-step dislocation form is used to perform high-dimensional representation of time series data in high-level measurement space. and The product of The linear transfer process describing the high-dimensional representation effectively captures the local transition rules of the current state in the measurement space.

[0076] However, when ψ is underfitted, the low-quality measurement space may lead to numerical instability or guide the model to converge in the wrong direction. To alleviate this problem and capture the globally shared dynamics, a learnable global Koopman operator is proposed. The learnable part can effectively construct the Koopman operator according to the current parameters, neutralizing the extreme case. For the overfitting problem in suboptimal space, It can represent the local dynamic process. It can represent globally shared dynamics. The linear dynamic system described by the Koopman operator is represented by the operator Method construction:

[0077]

[0078] By projecting and generating time series data using the Koopman operator, a biased linear dynamic system can be effectively constructed, ultimately outputting residual information and a high-dimensional representation on the prediction window. The residual information is expressed as is a high-dimensional representation on the look-back window.

[0079] After the Koopman operator has modeled the nonlinear time series into a linear dynamic system, the Kalman network based on the Kalman filter can further model and refine the uncertainty in the linear dynamic system described by the Koopman network, optimize the linear dynamic system, and adaptively adjust the system state. In the Kalman network, the residual information output by the Koopman network is first reused and integrated through the integrator:

[0080]

[0081] U=Integrator(Residual)=[u1,u2,…,u m ]

[0082] Next, describe the state transition of the system through the process model:

[0083] z k =Az k-1 +Bu k +w k

[0084]

[0085] in, is the state transition matrix, is the control input matrix, is the process noise, Q is the covariance matrix of the process noise, u k Belongs to the aforementioned integrated residual information U; z k It is the result of step k of the transfer process, and the initial state z0 is assigned to The last high-dimensional representation in

[0086] Then, by observing the model o k =Hz k +v k To update the system status. is the observation matrix, and the predicted value is considered as a prior observation in the update step. is the observation noise, and R is the covariance matrix of the observation noise. The goal in the Kalman network is to reuse the information from the nonlinear residuals and integrate it into the linear system constructed by the Koopman network, thereby obtaining a more accurate linear system and modeling uncertainty. In the Kalman network, all matrices are learnable. In addition, the covariance matrices Q and R are initialized to the identity matrix and the lower triangular matrix L is used. Q and L R Keep it positive: and

[0087] Then, the prediction step and update step are iterated. The prediction step can be expressed as:

[0088]

[0089] in, is the state transition matrix, is the control input matrix, is the state predicted at step k during the iteration, is the process uncertainty covariance matrix predicted in the kth step during the iteration, u k Belongs to the aforementioned integrated residual information U, z k-1 and P k-1 is the state in the k-1th step and the process uncertainty covariance matrix, and Q is the covariance matrix of the process noise.

[0090] Then, the update step is performed by the Kalman gain K k Measure the weight between observations and predictions, and update the linear system:

[0091]

[0092] Among them, z k and P k It is based on the Kalman gain K k The updated state vector and the covariance matrix of the process uncertainty; Kalman gain K k The process uncertainty covariance matrix predicted by the k-th step in the iterative process The observation matrix H and the covariance matrix R of the observation noise are calculated collaboratively.

[0093] Then, we obtain the updated state vector Z = [z1, z2, ..., z m ] and the covariance matrix of process uncertainty P = [P1, P2, ..., P m ], which describe the uncertainty of the time process in a dynamic system. In order to fully utilize the power of the integrator, a residual connection is introduced: Z′=Z+U, where Z′ is the predicted latent variable and U is the integration result of the residual information.

[0094] During the training process, the Kalman model uses an integrator to integrate nonlinear information and gradually adjust the topological structure of the measurement space. After optimization, the deviation of the linear system constructed by the Koopman network gradually decreases, making U → 0. This helps to form a linear dynamic system in the measurement space and gradually reduces the dependence on the integrator to obtain the predicted latent variables Z′ and the covariance matrix P of the process uncertainty.

[0095] The reconstruction loss is expressed as follows:

[0096]

[0097] Among them, X represents the input sequence, X Rec represents the sequence of Koopman network reconstructions.

[0098] After obtaining the predicted latent variable Z′ and the covariance matrix P of process uncertainty, the posterior distribution is varied Resampling is performed to ensure the propagation of gradients and make the model generalize. Finally, the decoder maps these resampled latent variables back to the original space and distributes them through the Gaussian distribution. To model the target distribution of the prediction, where μ and σ represent the mean and standard deviation respectively. Model the distribution of the target sequence Y After that, the prediction result of the target sequence Y can be obtained by sampling from it.

[0099] The input sequence embedding module, Koopman network, Kalman network and decoder in the present invention together constitute the K 2 The VAE model, by combining Koopman theory and Kalman filtering, adopts the framework of generative model VAE, aiming to solve the error accumulation and uncertainty modeling problems in the existing technology of long-term time series forecasting;

[0100] K 2 VAEs use a carefully designed process to model time series from the perspective of dynamic systems.

[0101] First, the input segment embedding module divides the time series into multiple segments and obtains their representation in the latent space.

[0102] Next, a Koopman network maps these segments into a measurement function space, where the inherent nonlinearities in time series and the complex joint distributions among variables are reconsidered to simplify modeling.

[0103] The Koopman operator is then used to fit these markers to construct a linear dynamic system. However, a linear system constructed from a perfect measurement function is an ideal goal, which means that the time series generated by the Koopman operator is biased.

[0104] The Kalman network can refine the biased linear system and model uncertainty by outputting the covariance matrix of the multidimensional state vector. In this way, the Kalman network effectively alleviates the error accumulation in the linear system and helps construct a variational posterior distribution with clear semantics in the measurement function space. Compared with the diffusion model or flow model with a longer generation process, K 2VAE adopts a VAE-based structure, consisting of a lightweight but efficient Koopman network and Kalman network, which can generate quickly in one step and reduce memory usage.

[0105] In addition to the reconstruction loss, the training optimization method of the time series probability prediction method of the present invention also includes the evidence lower bound loss, which is expressed as follows:

[0106]

[0107] in, Represents the empirical log-likelihood loss, which is used to optimize the posterior distribution The accuracy of Represents KL divergence, which is used to measure K 2 Variational distribution constructed in VAE latent space and the expected distribution the differences between;

[0108] Among them, ELBO loss ensures that K 2 The basic mechanism of the VAE model, and the reconstruction loss helps to measure the linearization process of the function ψ.

[0109] like Figure 3 As shown in the figure, in the short-step prediction scenario (input 24 steps and prediction 24 steps, Exchange-S is input 30 steps and prediction 30 steps), the red bold represents the best, the blue underline represents the suboptimal, the Model column represents different probability prediction models, the evaluation indicators are CRPS and NMAE, and the evaluation results on Exchange-S to ETTm2-S are shown; it can be seen from the figure that the method in the present invention has the best or suboptimal results in different evaluation indicators.

[0110] like Figure 4 As shown in the figure, in the long-step prediction (96 steps of input and 720 steps of prediction) scenario, the red bold line represents the best, the blue underline represents the suboptimal line, the mdel column represents different probability prediction models, and the evaluation indicators are CRPS and NMAE, and the evaluation results on ETTm1-L to ILI-L. It can be seen from the figure that the method in the present invention achieves the best results in different evaluation indicators.

[0111] like Figure 5 As shown in the figure, in the lightweight test scenario, the red bold line represents the best and the blue underline represents the suboptimal line. According to the inference speed (Inference Speed (seconds)) and memory usage (Memory (GB)), the cost of outputting a sample during inference of different models is compared. It can be seen from the figure that the method in the present invention almost always achieves the best or suboptimal results in different evaluation indicators.

[0112] Example 1

[0113] Short-term probability forecasting scenarios include but are not limited to financial forecasting scenarios. For example, the exchange rate dataset Exchange-S (total length 6071, with 8 variables) records the daily exchange rates of 8 countries from 1990 to 2016. In this dataset, it is usually necessary to use the past 30 steps to predict the exchange rate of each country. The input window predicts the next 30 steps The probability distribution on K 2 When VAE models it, each input window needs to be The final distribution is modeled by inputting the token embedding part, the Koopman part, the Kalman part, and the decoder part. During the inference process, sampling is performed based on the output target distribution to obtain the probability prediction value. The detailed steps are as follows:

[0114] Consider the nonlinear input time series from the Exchange-S dataset Divide the nonlinear input time series into multiple non-overlapping patches: Among them, 5 represents the patch size and 6 represents the number of patches;

[0115] Then, X P Mapped to a high-dimensional latent space: X P′ =Projection(Flatten(X P )); During the mapping process, patch X P First it is flattened into Then it is mapped to The high-dimensional embedding space of is used to fuse variable information and linearize nonlinear time series modeling problems in the real world.

[0116] The high-dimensional patch is then fed into K 2 The VAE is transformed from the measurement function constructed according to the Koopman theory into a linear dynamic system in the measurement function space. The measurement function is expressed as ψ: where X P* Represents a marker projected into the measurement space.

[0117] To capture the local transition rules, a single-step extended dynamic mode decomposition technique is used to efficiently fit the optimal local Koopman operator.

[0118]

[0119]

[0120] in, express The MP pseudo-inverse, and The single-step dislocation form is used to perform high-dimensional representation of time series data in high-level measurement space. and The product of The linear transfer process describing the high-dimensional representation effectively captures the local transition rules of the current state in the measurement space.

[0121] A learnable global Koopman operator The learnable part can effectively construct the Koopman operator according to the current parameters, neutralizing the extreme case. For the overfitting problem in suboptimal space, It can represent the local dynamic process. It can represent globally shared dynamics. The linear dynamic system described by the Koopman operator is represented by the operator Method construction:

[0122]

[0123] By projecting and generating time series data using the Koopman operator, a biased linear dynamic system can be effectively constructed, ultimately outputting residual information and a high-dimensional representation on the prediction window. The residual information is expressed as is a high-dimensional representation on the look-back window.

[0124] Then, in the Kalman network, the residual information output by the Koopman network is first reused and integrated through the integrator:

[0125]

[0126] U=Integrator(Residual)=[u1,u2,…,u6]

[0127] Next, describe the state transition of the system through the process model:

[0128] z k =Az k-1 +Bu k +w k

[0129]

[0130] in, is the state transition matrix, is the control input matrix, is the process noise, Q is the covariance matrix of the process noise, u k Belongs to the aforementioned integrated residual information U; z kIt is the result of step k of the transfer process, and the initial state z0 is assigned to The last high-dimensional representation in

[0131] By observing the model k =Hz k +v k To update the system status. is the observation matrix, and the predicted value Treated as prior observations in the update step;

[0132] The prediction step and update step are iterated. The prediction step can be expressed as:

[0133]

[0134] The update step is done by the Kalman gain K k Measure the weight between observations and predictions, and update the linear system:

[0135]

[0136] We obtain the updated state vector Z = [z1, z2, …, z6] for each marker and the process uncertainty covariance matrix P = [P1, P2, …, P6], which describe the uncertainty of the time process in the dynamic system. To fully utilize the power of the integrator, we introduce a residual connection: Z′ = Z + U, where Z is the predicted latent variable and U is the result of integrating the residual information.

[0137] After obtaining the predicted latent variable Z′ and the covariance matrix P of process uncertainty, the posterior distribution is varied Resampling is performed to ensure the propagation of gradients and make the model generalize. Finally, the decoder maps these resampled latent variables back to the original space and distributes them through the Gaussian distribution. To model the target distribution of the prediction, where μ and σ represent the mean and standard deviation respectively. Model the distribution of the target sequence Y After that, the prediction result of the target sequence Y can be obtained by sampling from it.

[0138] Example 2

[0139] Short-term probability prediction scenarios include but are not limited to energy prediction scenarios. For example, the energy dataset Solar-S (total length 7009, with 137 variables) records the solar energy production capacity of 137 sites. In this dataset, it is usually necessary to use the past 24 steps to predict the solar energy production capacity of 137 sites. The input window predicts the next 24 steps The probability distribution on K 2 When VAE models it, each input window needs to be The final distribution is modeled by inputting the token embedding part, the Koopman part, the Kalman part, and the decoder part. During the inference process, sampling is performed based on the output target distribution to obtain the probability prediction value. The detailed steps are as follows:

[0140] Consider the nonlinear input time series from the Solar-S dataset Divide the nonlinear input time series into multiple non-overlapping patches: Among them, 4 represents the patch size and 6 represents the number of patches;

[0141] Then, X P Mapped to a high-dimensional latent space: X P′ =Projection(Flatten(X P )); During the mapping process, patch X P First it is flattened into Then it is mapped to The high-dimensional embedding space of is used to fuse variable information and linearize nonlinear time series modeling problems in the real world.

[0142] The high-dimensional patch is then fed into K 2 The VAE is transformed from the measurement function constructed according to the Koopman theory into a linear dynamic system in the measurement function space. The measurement function is expressed as ψ: where X P* Represents a marker projected into the measurement space.

[0143] To capture the local transition rules, a single-step extended dynamic mode decomposition technique is used to efficiently fit the optimal local Koopman operator.

[0144]

[0145] in, express The MP pseudo-inverse, and The single-step dislocation form is used to perform high-dimensional representation of time series data in high-level measurement space. and The product of The linear transfer process describing the high-dimensional representation effectively captures the local transition rules of the current state in the measurement space.

[0146] A learnable global Koopman operator κ glo The learnable part can effectively construct the Koopman operator according to the current parameters, neutralizing the extreme case κ loc For the overfitting problem in suboptimal space, It can represent the local dynamic process. It can represent globally shared dynamics. The linear dynamic system described by the Koopman operator is represented by the operator Method construction:

[0147]

[0148] By projecting and generating time series data using the Koopman operator, a biased linear dynamic system can be effectively constructed, ultimately outputting residual information and a high-dimensional representation on the prediction window. The residual information is expressed as is a high-dimensional representation on the look-back window.

[0149] Then, in the Kalman network, the residual information output by the Koopman network is first reused and integrated through the integrator:

[0150]

[0151] U=Integrator(Residual)=[u1,u2,…,u6]

[0152] Next, describe the state transition of the system through the process model:

[0153] z k =Az k-1 +Bu k +w k

[0154]

[0155] in, is the state transition matrix, is the control input matrix, is the process noise, Q is the covariance matrix of the process noise, u k Belongs to the aforementioned integrated residual information U; z k It is the result of step k of the transfer process, and the initial state z0 is assigned to The last high-dimensional representation in

[0156] By observing the model k =Hz k +v k To update the system status. is the observation matrix, and the predicted value Treated as prior observations in the update step;

[0157] The prediction step and update step are iterated. The prediction step can be expressed as:

[0158]

[0159] The update step is done by the Kalman gain K k Measure the weight between observations and predictions, and update the linear system:

[0160]

[0161] Obtain the updated state vector Z = [z1, z2, ..., z6] for each tag and the covariance matrix P = [ P 1, P2, …, P6], which describe the uncertainty of the time process in the dynamic system. In order to fully utilize the power of the integrator, a residual connection is introduced: Z′ = Z + U, where Z′ is the predicted latent variable and U is the integration result of the residual information.

[0162] After obtaining the predicted latent variable Z′ and the covariance matrix P of process uncertainty, the posterior distribution is varied Resampling is performed to ensure the propagation of gradients and make the model generalize. Finally, the decoder maps these resampled latent variables back to the original space and distributes them through the Gaussian distribution. To model the target distribution of the prediction, where μ and σ represent the mean and standard deviation respectively. Model the distribution of the target sequence Y After that, the prediction result of the target sequence Y can be obtained by sampling from it.

[0163] Example 3

[0164] Long-step probability prediction scenarios include but are not limited to predictions based on system sensor data. For example, the transformer system dataset ETTh1-L (total length 17420, with 7 variables) records the load characteristics of seven oil and power transformers from July 2016 to July 2018. The input window predicts the next 720 steps The probability distribution on K 2 When VAE models it, each input window needs to be The final distribution is modeled by inputting the token embedding part, the Koopman part, the Kalman part, and the decoder part. During inference, sampling is performed based on the output target distribution. The detailed steps are as follows:

[0165] Consider the nonlinear input time series from the Exchange-S dataset Divide the nonlinear input time series into multiple non-overlapping patches: Among them, 12 represents the patch size, 8 represents the number of patches,

[0166] Then, X P Mapped to a high-dimensional latent space: X P′ =Projection(Flatten(X P )); During the mapping process, patch X P First it is flattened into Then it is mapped to The high-dimensional embedding space of is used to fuse variable information and linearize nonlinear time series modeling problems in the real world.

[0167] The high-dimensional patch is then fed into K 2 The VAE is transformed from the measurement function constructed according to the Koopman theory into a linear dynamic system in the measurement function space. The measurement function is expressed as ψ: where X P* Represents a marker projected into the measurement space.

[0168] To capture the local transition rules, a single-step extended dynamic mode decomposition technique is used to efficiently fit the optimal local Koopman operator.

[0169]

[0170] in, express The MP pseudo-inverse, and The single-step dislocation form is used to perform high-dimensional representation of time series data in high-level measurement space. and The product of The linear transfer process describing the high-dimensional representation effectively captures the local transition rules of the current state in the measurement space.

[0171] A learnable global Koopman operator The learnable part can effectively construct the Koopman operator according to the current parameters, neutralizing the extreme case. For the overfitting problem in suboptimal space, It can represent the local dynamic process. It can represent globally shared dynamics. The linear dynamic system described by the Koopman operator is represented by the operator Method construction:

[0172]

[0173] By projecting and generating time series data using the Koopman operator, a biased linear dynamic system can be effectively constructed, ultimately outputting residual information and a high-dimensional representation on the prediction window. The residual information is expressed as is a high-dimensional representation on the look-back window.

[0174] Then, in the Kalman network, the residual information output by the Koopman network is first reused and integrated through the integrator:

[0175]

[0176] U=Integrator(Residual)=[u1,u2,…,u8]

[0177] Next, describe the state transition of the system through the process model:

[0178] z k =Az k-1 +Bu k +w k

[0179]

[0180] in, is the state transition matrix, is the control input matrix, is the process noise, Q is the covariance matrix of the process noise, u k Belongs to the aforementioned integrated residual information U; z k It is the result of step k of the transfer process, and the initial state z0 is assigned to The last high-dimensional representation in

[0181] By observing the model k =Hz k +v k To update the system status. is the observation matrix, and the predicted value Treated as prior observations in the update step;

[0182] The prediction step and update step are iterated. The prediction step can be expressed as:

[0183]

[0184] The update step is done by the Kalman gain K k Measure the weight between observations and predictions, and update the linear system:

[0185]

[0186] Get the updated state vector Z = [z1, z2, ..., z 60] and the covariance matrix of process uncertainty P = [P1, P2, ..., P 60 ], which describe the uncertainty of the time process in a dynamic system. In order to fully utilize the power of the integrator, a residual connection is introduced: Z′=Z+U, where Z′ is the predicted latent variable and U is the integration result of the residual information.

[0187] After obtaining the predicted latent variable Z′ and the covariance matrix P of process uncertainty, the posterior distribution is varied Resampling is performed to ensure the propagation of gradients and make the model generalize. Finally, the decoder maps these resampled latent variables back to the original space and distributes them through the Gaussian distribution. To model the target distribution of the prediction, where μ and σ represent the mean and standard deviation respectively. Model the distribution of the target sequence Y After that, the prediction result of the target sequence Y can be obtained by sampling from it.

[0188] Example 4

[0189] Long-step probability prediction scenarios include but are not limited to predictions for traffic data, such as the traffic dataset Traffic-L (total length 17544, with 862 variables), which records hourly road occupancy data recorded by San Francisco highway sensors from 2015 to 2016. The input window predicts the next 720 steps The probability distribution on K 2 When VAE models it, each input window needs to be The final distribution is modeled by inputting the token embedding part, the Koopman part, the Kalman part, and the decoder part. During inference, sampling is performed based on the output target distribution. The detailed steps are as follows:

[0190] Consider the nonlinear input time series from the Traffic-L dataset Divide the nonlinear input time series into multiple non-overlapping patches: Among them, 12 represents the patch size, 8 represents the number of patches,

[0191] Then, X P Mapped to a high-dimensional latent space: X P′ =Projection(Flatten(X P )); During the mapping process, patch X P First it is flattened into Then it is mapped to The high-dimensional embedding space of is used to fuse variable information and linearize nonlinear time series modeling problems in the real world.

[0192] The high-dimensional patch is then fed into K 2 The VAE is transformed from the measurement function constructed according to the Koopman theory into a linear dynamic system in the measurement function space. The measurement function is expressed as ψ: where X P* Represents a marker projected into the measurement space.

[0193] To capture the local transition rules, a single-step extended dynamic mode decomposition technique is used to efficiently fit the optimal local Koopman operator.

[0194]

[0195] in, express The MP pseudo-inverse, and The single-step dislocation form is used to perform high-dimensional representation of time series data in high-level measurement space. and The product of The linear transfer process describing the high-dimensional representation effectively captures the local transition rules of the current state in the measurement space.

[0196] A learnable global Koopman operator The learnable part can effectively construct the Koopman operator according to the current parameters, neutralizing the extreme case. For the overfitting problem in suboptimal space, It can represent the local dynamic process. It can represent globally shared dynamics. The linear dynamic system described by the Koopman operator is represented by the operator Method construction:

[0197]

[0198] By projecting and generating time series data using the Koopman operator, a biased linear dynamic system can be effectively constructed, ultimately outputting residual information and a high-dimensional representation on the prediction window. The residual information is expressed as is a high-dimensional representation on the look-back window.

[0199] Then, in the Kalman network, the residual information output by the Koopman network is first reused and integrated through the integrator:

[0200]

[0201] U=Integrator(Residual)=[u1,u2,…,u8]

[0202] Next, describe the state transition of the system through the process model:

[0203] z k =Az k-1 +Bu k +w k

[0204]

[0205] in, is the state transition matrix, is the control input matrix, is the process noise, Q is the covariance matrix of the process noise, v k Belongs to the aforementioned integrated residual information U; z k It is the result of step k of the transfer process, and the initial state z0 is assigned to The last high-dimensional representation in

[0206] By observing the model k =Hz k +v k To update the system status. is the observation matrix, and the predicted value Treated as prior observations in the update step;

[0207] The prediction step and update step are iterated. The prediction step can be expressed as:

[0208]

[0209] The update step is done by the Kalman gain K k Measure the weight between observations and predictions, and update the linear system:

[0210]

[0211] Get the updated state vector Z = [z1, z2, ..., z 60 ] and the covariance matrix of process uncertainty P = [P1, P2, ..., P 60 ], which describe the uncertainty of the time process in a dynamic system. In order to fully utilize the power of the integrator, a residual connection is introduced: Z′=Z+U, where Z′ is the predicted latent variable and U is the integration result of the residual information.

[0212] After obtaining the predicted latent variable Z′ and the covariance matrix P of process uncertainty, the posterior distribution is varied Resampling is performed to ensure the propagation of gradients and make the model generalize. Finally, the decoder maps these resampled latent variables back to the original space and distributes them through the Gaussian distribution. To model the target distribution of the prediction, where μ and σ represent the mean and standard deviation respectively. Model the distribution of the target sequence Y After that, the prediction result of the target sequence Y can be obtained by sampling from it.

[0213] References

[0214] [1]Cheng, Cheng, Jia Ding, and Yong Zhang. "A Koopman operator approach for machinery health monitoring and prediction with noisy and low-dimensional industrial time series." Neurocomputing 406(2020):204-214.

[0215] [2]Liu, Yong, et al. "Koopa: Learning non-stationary time series dynamics with koopman predictors." Advances in neural information processing systems 36(2023):12271-12290.

[0216] [3] Lange, Henning, Steven L. Brunton, and J. Nathan Kutz. "From Fourier to Koopman: Spectral methods for long-term time series prediction." Journal of Machine Learning Research 22.41 (2021): 1-38.

[0217] The protection content of the present invention is not limited to the above embodiments. Without departing from the spirit and scope of the present invention, changes and advantages that can be thought of by those skilled in the art are included in the present invention and are protected by the appended claims.

Claims

1. A time series probability prediction method based on Koopman theory and Kalman filtering, characterized in that: The method comprises the following steps: Step 1: Patch the input time series data and map it into a high-dimensional latent space; Step 2: Map the patched time series data into a high-dimensional measurement space and model the linear dynamic system using the Koopman operator; Step 3: Optimize the linear dynamic system using a Kalman network, model the state of the linear dynamic system, update the state estimate, and output potential variables; Step 4: Resample from the variational posterior distribution, map the latent variables back to the original space, model and predict the target distribution, and output the probabilistic prediction results of the time series.

2. The prediction method according to claim 1, wherein: In step 1, the input time series data is patched Divide into one or more non-overlapping patches Where s represents the patch size, n represents the number of patches, represents the i-th patch; Patch X P Flattened to Mapping to a high-dimensional embedding space through linear projection Indicated as X P′ =Projection(Flatten(X P )).

3. The prediction method according to claim 1, wherein: In step 2, the learnable multilayer perceptron network is used as the measurement function to map the input patched time series data into a high-dimensional measurement space, which is expressed as Perform linearization processing; where ψ is the measurement function, Markers representing projections into the measurement space; and / or, The Koopman operator for constructing the linear dynamic system includes the local Koopman operator and the global Koopman operator The complete Koopman operator is expressed as The local Koopman operator The single-step extended dynamic mode decomposition technique is used to fit the following: in, express The MP pseudo-inverse, and The single-step dislocation form is used to perform high-dimensional representation of time series data in high-level measurement space. and The product of Describe the linear transfer process of high-dimensional representations; The linear dynamic system is modeled in the measurement space by an infinite-dimensional linear Koopman operator, which is expressed as follows: Output residual information and high-dimensional representation on the prediction window, expressed as: Residual information: High-dimensional representation over the prediction window: in, 4. The prediction method according to claim 1, wherein: In step 3, the nonlinear residual information is reused and integrated into the linear dynamic system constructed by the Koopman network to optimize the linear dynamic system; and / or, The residual information is utilized and integrated through the integrator, which is expressed as follows: U=Integrator(Residual)=[u1,u2,…,u m ], Among them, Residual is the residual information output by the Koopman network, which is represented by the actual P* Representation of reconstruction with the Koopman operator The difference is obtained; and / or, Then the state transition of the system is described by the process model, which is expressed as follows: <h2 style=";text-align:left;direction:ltr">z<h2 style=";text-align:left;direction:ltr"> k <h2 style=";text-align:left;direction:ltr"> =Az<h2 style=";text-align:left;direction:ltr"> k-1 <h2 style=";text-align:left;direction:ltr"> +Bu<h2 style=";text-align:left;direction:ltr"> k <h2 style=";text-align:left;direction:ltr"> +w<h2 style=";text-align:left;direction:ltr"> k <h2 style=";text-align:left;direction:ltr"> , in, is the state transition matrix, is the control input matrix, is the process noise, Q is the covariance matrix of the process noise; u k Belongs to the integrated residual information U; z k It is the result of step k of the transfer process, and the initial state z0 is assigned to The last high-dimensional representation in and / or, By observing the model k =Hz k +v k Update the system status, including: is the observation matrix, and the predicted value is considered as a prior observation in the update step, is the observation noise, R is the covariance matrix of the observation noise; and / or, The prediction step and the update step are iterated to update the linear dynamic system.

5. The prediction method according to claim 4, wherein: The prediction step is expressed as: in, is the state transition matrix, is the control input matrix, is the state predicted at step k during the iteration, is the process uncertainty covariance matrix predicted in the kth step during the iteration; u k Belongs to the integrated residual information U, z k-1 and P k-1 is the state in the k-1th step and the process uncertainty covariance matrix, Q is the covariance matrix of the process noise; and / or, The update step is done by the Kalman gain K k Measure the weight between observations and predictions, and update the linear system: Among them, z k and P k It is based on the Kalman gain K k The updated state vector and the covariance matrix of the process uncertainty; Kalman gain K k The process uncertainty covariance matrix predicted by the k-th step in the iterative process The observation matrix H and the covariance matrix R of the observation noise are calculated collaboratively.

6. The prediction method according to claim 4, wherein: By initializing the covariance matrix of process noise and the covariance matrix of observation noise to the identity matrix and using a lower triangular matrix to maintain positive definiteness, it is expressed as follows: and 7. The prediction method according to claim 1, wherein: In step 4, the predicted latent variable Z′ and the covariance matrix P of the process uncertainty are obtained and the posterior distribution is varied. Resample and map the resampled latent variables back to the original space through Gaussian distribution To model the predicted target distribution, sample Y from the target distribution to complete the probability prediction; μ and σ represent the mean and standard deviation respectively.

8. A prediction system for implementing the prediction method according to any one of claims 1 to 7, characterized in that: The prediction system includes: a data preprocessing module, a Koopman network module, a Kalman network module, a decoder module, and a prediction result output module; The data preprocessing module is used to receive time series data and perform segmentation processing on it, dividing the time series data into multiple segments and mapping them into a high-dimensional latent space, and extracting feature representations of the time series; The Koopman network module is used to map time series data to a high-dimensional measurement space based on the Koopman theory, and to construct a linear dynamic system using the Koopman operator to achieve linear modeling of the time series; The Kalman network module is used to optimize the linear dynamic system generated by the Koopman network based on the Kalman filter, model the state of the linear dynamic system, update the state estimate, and output the potential variables; The decoder module is used to resample from the variational posterior distribution, map the latent variables back to the original space, model and predict the target distribution, and obtain a probabilistic prediction result of the time series; The prediction result output module is used to output the probability prediction result of the time series.

9. Application of the prediction method according to any one of claims 1 to 7, or the prediction system according to claim 8, in time series probability prediction in both short-step probability prediction scenarios and long-step probability prediction scenarios.

Images