A time series data prediction method, system, device and medium

Abstract

The application provides a time series data prediction method, system, device and medium, and belongs to the technical field of data processing. The method comprises the following steps: preprocessing multi-source original time series data to generate a training data set containing historical time series data blocks; establishing a linear Gaussian state space model, and defining a key parameter matrix of the model as a time-varying parameter matrix; utilizing a deep parameter generation network model to dynamically generate a time-varying parameter matrix set according to the historical time series data blocks; coupling the deep parameter generation network model and a Kalman filtering algorithm into an end-to-end differentiable system, performing differentiable Kalman filtering on the training set to estimate and predict the state, and optimizing the model by utilizing loss function gradient back propagation; and dynamically generating the time-varying parameter matrix by utilizing the optimized model and performing multi-step forward prediction to output a probability prediction result. The application realizes nonlinear adaptive modeling and uncertainty quantification in a single framework, and improves the accuracy and interpretability of prediction.

Description

Technical Field

[0001] This invention belongs to the field of data processing technology, and more specifically relates to a method, system, device and medium for time series data prediction. Background Technology

[0002] Time series forecasting is one of the core challenges in data science. Its goal is to infer future trends from noisy historical observation data, and it has wide applications in business intelligence, enterprise management, financial risk control, energy dispatching, industrial control, and medical diagnostics. For example, in intelligent supply chain scenarios, the system needs to combine historical outbound volumes, seasonal factors, promotional plans, and macroeconomic indicators to predict future inventory demand and quantify the confidence interval of replenishment recommendations to balance service levels and inventory costs. In large-scale group asset operation and maintenance scenarios, it is necessary to dynamically model the time-series data such as vibration and temperature collected by equipment sensors and predict and output health indices for future periods, assisting the operation and maintenance team in identifying potential anomalies in advance and arranging preventative maintenance plans.

[0003] For a long time, the technological evolution in this field has mainly followed two paths: one is traditional statistical learning methods represented by the Autoregressive Integrated Moving Average (ARIMA) model and its variants; the other is modern deep learning models represented by Long Short-Term Memory (LSTM) networks, Transformer, and so on. Both types of methods have their advantages, but each also faces insurmountable theoretical and practical bottlenecks.

[0004] Traditional statistical models (such as ARIMA, state-space models, and linear quadratic Gaussian control frameworks) are built upon rigorous mathematical assumptions, possessing transparent structures and good interpretability. Taking the linear quadratic Gaussian control framework as an example, its core component, the Kalman filter, can recursively and efficiently fuse new observation data and provide the mean and covariance matrices of the state estimate, thus naturally providing a measure of the uncertainty of the estimation results—crucial for risk-sensitive applications. However, the fatal flaw of these methods lies in their strict linear assumptions and fixed prior settings. Real-world time series data commonly exhibit nonlinearity, nonstationarity, time-varying characteristics, and complex nonlinear dynamics. When significant non-Gaussian noise, outliers, or complex patterns exist in the data, the predictive performance of traditional statistical models deteriorates sharply, failing to adapt to the dynamic changes in real-world scenarios.

[0005] On the other hand, deep learning models (such as LSTM and Transformer) have demonstrated powerful pattern recognition and feature learning capabilities through multi-layer nonlinear transformations, achieving breakthrough results in many prediction tasks. However, the "black box" nature of these models brings significant drawbacks. First, their predictions are typically deterministic point estimates, failing to provide any quantitative indicators of prediction reliability, such as prediction confidence intervals or uncertainty estimates. This poses a significant obstacle to their application in fields with zero tolerance for errors, such as financial risk control and medical diagnosis. Second, deep learning models are purely data-driven, making it difficult to effectively incorporate existing domain knowledge or physical laws. Once trained, their model parameters remain fixed, unable to adaptively adjust the model structure or parameters according to the real-time state of the system, limiting their generalization ability and interpretability in non-stationary environments.

[0006] In summary, a significant gap exists in the current technological landscape: while traditional statistical models offer advantages in uncertainty measurement and structural transparency, their linear assumptions prevent them from capturing complex nonlinear dynamics; deep learning methods, despite their powerful nonlinear fitting capabilities, lack uncertainty quantification and model adaptability, and struggle to integrate domain knowledge. While there are preliminary explorations combining deep learning with Kalman filtering (such as using neural networks to estimate the static parameters of Kalman filters), these parameters are fixed once trained and cannot be dynamically adjusted based on real-time changes in the system state. Furthermore, the training processes of the network and the filter are typically independent, making end-to-end joint optimization difficult. To date, no solution has been found that can simultaneously achieve data-driven dynamic generation of time-varying parameters, differentiable end-to-end joint optimization, uncertainty quantification of prediction results, and structured interpretability of model parameters within a single framework. Summary of the Invention

[0007] To address the above problems, the present invention aims to provide a time series data prediction method, system, device, and medium. By coupling a deep parameter generation network model with a differentiable Kalman filter algorithm end-to-end, the deep network dynamically generates a time-varying parameter matrix to drive a linear quadratic Gaussian control framework, thereby achieving adaptive and interpretable time series prediction that integrates the nonlinear fitting capabilities of deep learning with the advantages of uncertainty quantification in control theory.

[0008] To achieve the above objectives, the present invention employs the following technical solution: In a first aspect, embodiments of this application provide a time series data prediction method, including: The collected multi-source raw time-series data are preprocessed to generate a standardized training dataset, which includes historical time-series data blocks used for model training. A linear Gaussian state-space model is established, and the key parameter matrix in the linear Gaussian state-space model is defined as a time-varying parameter matrix, forming a dynamic parameterized model framework. A deep parameter generation network model is used to dynamically generate a set of time-varying parameter matrices for the current system state based on historical time-series data blocks in the training dataset and in accordance with a dynamic parameterization model framework. The set of time-varying parameter matrices includes a state transition matrix, an observation matrix, a process noise covariance matrix, and an observation noise covariance matrix. The deep parameter generation network model is coupled with the Kalman filter algorithm into an end-to-end differentiable system. The differentiable Kalman filter algorithm is executed on the training dataset for state estimation and prediction. The loss function is calculated based on the prediction error. The gradient of the loss function is backpropagated to optimize the deep parameter generation network model with the goal of maximizing prediction accuracy, resulting in the optimized deep parameter generation network model. The optimized depth parameter generation network model is used to dynamically generate a time-varying parameter matrix from the latest observation data, and multi-step forward prediction is performed based on the time-varying parameter matrix to output the probability prediction result.

[0009] In an optional implementation, the preprocessing of the collected multi-source raw time-series data to generate a standardized training dataset includes: Receive raw data collected from multiple channels, including core time-series data, external covariate data, and metadata; perform a unified timestamp operation on the raw data and align all data to the same time granularity to generate an aligned time-series dataset; Missing value detection is performed on the aligned time series dataset, and missing values ​​are processed using a time series feature-based method. After missing value imputation, a complete dataset is generated. Outlier detection is performed on the complete dataset, using a Z-score method combined with a sliding window or an isolated forest algorithm to identify outliers; the detected outliers are corrected or removed, and anomaly labels are generated and added to the complete dataset to generate a corrected dataset. Perform differencing operations or relative rate of change calculations on the non-stationary sequences in the corrected dataset to ensure the stationarity of the sequences and generate a stationary dataset; Perform advanced feature engineering operations on the stabilized dataset, specifically including: For scenarios containing high-frequency sampled data, aggregated statistical features are calculated based on the high-frequency data and used as the observations of the model. Set a time index Indicates the first Each sampling period, index of time intervals within the period ,in This represents the total number of intervals within the current period. For the first Within the first cycle The rate of change of the variable over time intervals; using the formula Calculate aggregate statistical features; Calculate moving average features, relative strength index features, and Bollinger Band width features as supplementary features, and create interaction term features and lag term features between different variables; A feature sequence set is constructed based on aggregate statistical features, moving average features, relative strength index features, Bollinger Band width features, interaction term features, and lag term features. Z-score normalization is performed independently on each feature sequence in the feature sequence set. The normalized feature data is then divided into sliding segments according to a fixed time window length to construct multiple historical time series data blocks with consecutive time steps. Each historical time series data block contains normalized feature data with multiple consecutive time steps. Historical time-series data blocks are integrated into a normalized training dataset.

[0010] In an optional implementation, the step of establishing a linear Gaussian state-space model and defining the key parameter matrix in the linear Gaussian state-space model as a time-varying parameter matrix to form a dynamic parameterized model framework includes: Based on the training dataset, a linear Gaussian state-space model suitable for time series data prediction is established, including the state equation and the observation equation: The state equation is Its process noise ; The observation equation is Its observation noise ; in, For discrete-time indexing; for dimensional state vector, for 3D observation vector, for The dimension-controlled input vector includes external covariates; for The state transition matrix; for The control input matrix; for The observation matrix; for A process noise vector of dimension , which follows a mean of zero and a covariance of . Gaussian distribution; for The observed noise vector has a mean of zero and a covariance of 1. Gaussian distribution; for The process noise covariance matrix; for The observation noise covariance matrix; For all key parameter matrices in the linear Gaussian state-space model structure , , , , The time-varying definition operation is executed, setting it to be time-varying and data-driven, generating a time-varying parameter matrix definition; where, Used to reflect the dynamic changes within the system. Used to characterize the degree of influence of external factors on the system state. Used to describe the mapping relationship from hidden states to observed values. Used to represent the inherent uncertainty of a system. Used to represent the measurement error level; outputs the definition of a time-varying parameter matrix; The structure and dimension definition operations are performed on the time-varying parameter matrix to form a dynamic parameterized model framework.

[0011] In an optional implementation, the step of utilizing a deep parameter generation network model to dynamically generate a set of time-varying parameter matrices for the current system state, based on historical time-series data blocks in the training dataset and following a dynamic parameterization model framework, includes: Extract historical time-series data blocks from the training dataset; A multi-level, multi-path encoder-decoder structure deep parameter generation network model is constructed, and the output structure of the deep parameter generation network model is determined by a dynamic parameterized model framework. The historical time series data blocks are input into the depth parameters to generate a time series feature encoder for the network model, which performs time series dependency capture operations in both forward and backward directions to generate a hidden state sequence; the time series feature encoder adopts a bidirectional long short-term memory network or a Transformer encoder. Pooling is performed on the hidden state sequence, and the final hidden state of the temporal feature encoder or the sequence representation aggregated by the pooling layer is used as the context encoding vector. The context encoding vector is input into the multi-branch parameter generation head of the deep parameter generation network model. The multi-branch parameter generation head has multiple parallel fully connected layer branches. The parameter generation operation of the corresponding matrix is ​​performed through each fully connected layer branch to generate the parameter matrix. The fully connected layer branches include: State transition matrix The generation branch is used to perform a fully connected mapping operation on the context encoding vector, outputting a vector that is reshaped to a dimension of . matrix ; Observation matrix The generation branch is used to perform a fully connected mapping operation on the context encoding vector, and the generation dimension is... matrix ; Noise covariance matrix and The generation branch is used to perform a fully connected mapping operation on the context encoding vector to generate its Cholesky factor, denoted as... for The Cholesky decomposition of the lower triangular matrix satisfies ,remember for The Cholesky decomposition of the lower triangular matrix satisfies ; Perform an integration operation on the parameter matrix to form a set of dynamic system parameter matrices. .

[0012] In an optional implementation, the deep parameter generation network model is coupled with the Kalman filter algorithm as an end-to-end differentiable system. The differentiable Kalman filter algorithm is executed on the training dataset for state estimation and prediction. A loss function is calculated based on the prediction error. The gradient of the loss function is used for backpropagation to optimize the deep parameter generation network model with the goal of maximizing prediction accuracy, resulting in an optimized deep parameter generation network model. This includes: Based on the set of dynamic system parameter matrices, the Kalman filter algorithm is transformed into a differentiable Kalman filter module, and the iterative formulas for the prediction step and the update step are defined. The iterative formula for the prediction step includes:

[0013]

[0014]

[0015] in, For discrete-time indexing; According to the date Time information Prior estimation of the state at time step; As of Time information Posterior estimation of the state at time step; According to the date Time information Prior estimates of time-based observations; Let be the covariance matrix of the prior state estimate; The iterative formula for the update step includes:

[0016]

[0017]

[0018] in, Here is the covariance matrix of the posterior state estimate. Here is the Kalman gain matrix. It is an identity matrix with the same dimensions as the state vector; for The actual observed value at time; Let the length of the history window be... The prediction window length is Total time step Before use A set of data and dynamic system parameter matrices at each time step The Kalman filter update operation is performed step by step using the differentiable Kalman filter module to initialize the Kalman filter state and generate the initial state estimate. After use Multi-step prediction and loss calculation operations are performed on data at each time step; the parameter matrix used in the prediction is uniformly set to... Using loss function Calculate the loss function value; ,express The total uncertainty covariance matrix of the observations at time points; Representation matrix The determinant; This is the transpose of the prediction error vector; for The inverse matrix; Perform gradient calculation on the loss function value, and calculate the gradient of the loss function with respect to the weights of the depth parameter generation network model using the backpropagation algorithm; use the calculated gradient to perform weight update operation, and update the weights of the depth parameter generation network model; The updated weights are loaded into the deep parameter generation network model to generate an optimized deep parameter generation network model.

[0019] In an optional implementation, the step of dynamically generating a time-varying parameter matrix from the latest observation data using the optimized depth parameter generation network model, and performing multi-step forward prediction based on the time-varying parameter matrix to output a probability prediction result includes: The optimized depth parameters are used to generate a network model. Forward propagation is performed based on historical time-series data blocks of the latest observation data to dynamically generate a set of time-varying parameter matrices. A multi-step forward prediction operation is performed based on the filtered state distribution. Let the current time be... Predicting the future step, , To generate the observation predictions and prediction covariance by performing the following calculations, based on the preset maximum number of prediction steps:

[0020]

[0021]

[0022]

[0023] in, According to the date Time information Prediction of the state at any given moment; Here is the covariance matrix for state prediction; According to the date Time information Prediction of time-based observations; The covariance matrix of the predicted observations; , , , The future time-time parameter matrix is ​​generated by the optimized depth parameter generation network model; Based on the observed values ​​and predicted covariance, a prediction distribution construction operation is performed to provide future observed values. Predicted distribution Generate probability prediction results; Set the credit level to ,in At the significance level, For standard normal distribution Quantiles; based on probability prediction results, confidence interval calculation is performed using the formula... Calculate the confidence interval for each observed variable; where, Indicates taking the matrix A vector consisting of the diagonal elements; Based on the time-varying parameter matrix set, the state transition matrix is ​​analyzed to understand the system's stability and main change patterns through its eigenvalues ​​and eigenvectors; the observation matrix is ​​analyzed to understand the mapping relationship between each observed variable and the hidden state; the process noise covariance matrix and the observation noise covariance matrix are analyzed, and their numerical values ​​are compared to determine whether the uncertainty mainly originates from within the system or from the observation process. An interpretability analysis report is generated based on the analysis results.

[0024] In an optional implementation, the state transition matrix Branch generation is achieved by generating branches. The matrix is ​​constrained so that the modulus of its eigenvalues ​​is less than 1 to ensure the dynamic stability of the system; a gating mechanism or adaptive attention is introduced into the temporal feature encoder of the deep parameter generation network model to automatically adjust the weights or biases of each parameter generation branch according to the features of the current input sequence.

[0025] Secondly, embodiments of this application also provide a time series data prediction system, including: The data preprocessing module is used to preprocess the collected multi-source raw time-series data to generate a standardized training dataset, which includes historical time-series data blocks for model training. The dynamic system model building module is used to build a linear Gaussian state-space model and define the key parameter matrix in the linear Gaussian state-space model as a time-varying parameter matrix, forming a dynamic parameterized model framework. The deep parameter generation module is used to generate a network model using deep parameters. Based on the historical time-series data blocks in the training dataset, it dynamically generates a set of time-varying parameter matrices for the current system state according to the dynamic parameterization model framework. The set of time-varying parameter matrices includes a state transition matrix, an observation matrix, a process noise covariance matrix, and an observation noise covariance matrix. The differentiable filtering and optimization module is used to couple the depth parameter generation network model and the Kalman filter algorithm into an end-to-end differentiable system. It performs state estimation and prediction on the training dataset, calculates the loss function based on the prediction error, and uses the gradient of the loss function for backpropagation to optimize the depth parameter generation network model with the goal of maximizing prediction accuracy, thereby obtaining the optimized depth parameter generation network model. The prediction reasoning and interpretability analysis module is used to dynamically generate a time-varying parameter matrix from the latest observation data using the optimized deep parameter generation network model, and to perform multi-step forward prediction based on the time-varying parameter matrix, outputting the probability prediction result.

[0026] Thirdly, embodiments of this application also provide an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps of the time series data prediction method as described in any of the above.

[0027] Fourthly, embodiments of this application also provide a storage medium storing a computer program thereon, which, when executed by a processor, implements the steps of the time series data prediction method as described in any of the above claims.

[0028] As can be seen from the above technical solutions, the present invention has the following advantages: The time series data prediction method provided in this application firstly utilizes a deep parameter generation network model to dynamically generate time-varying parameter matrices based on historical time series data, breaking through the linear assumptions and fixed parameter limitations of the traditional linear quadratic Gaussian control framework, and achieving adaptive modeling of the dynamic characteristics of complex nonlinear and non-stationary systems. Secondly, a differentiable Kalman filter module is integrated as a trainable component into the end-to-end optimization process, enabling the model to inherit the optimal state estimation characteristics of Kalman filtering while jointly optimizing the deep parameter generation network through gradient backpropagation, thereby simultaneously obtaining the powerful nonlinear fitting ability of deep learning and the rigorous uncertainty quantification ability of control theory. Thirdly, by outputting the prediction distribution and confidence interval and providing structured analysis of the state transition matrix, observation matrix, and noise covariance matrix, a quantitative evaluation of the reliability of the prediction results and the interpretability of the model decision-making process are achieved. Finally, through online learning and adaptive mechanisms, the incremental updates of the model are supported, enabling continuous adaptation to changes in data distribution. In summary, this application simultaneously achieves nonlinear fitting, uncertainty quantification, interpretability, and adaptive evolution within a single framework, effectively overcoming the technical challenge of the coexistence of the linear assumption limitations of existing statistical models and the "black box" defects of deep learning models. It provides a time series data prediction solution with accuracy, reliability, and interpretability for fields such as financial risk control, energy dispatch, and industrial control.

[0029] This application constructs a deep parameter generation network model to dynamically generate a set of time-varying parameter matrices based on historical time-series data blocks. This transforms the fixed parameter matrix in the traditional linear quadratic Gaussian control framework into dynamic parameters that are driven by data and adjusted in real time according to the system state. This enables the originally strictly linear state-space model to indirectly fit the nonlinear and non-stationary dynamic characteristics that are widely present in the real world through time-varying parameters, effectively overcoming the technical defect of the traditional statistical model that the prediction performance drops sharply when facing complex nonlinear systems.

[0030] This application transforms the Kalman filter algorithm into a differentiable module, coupling it with a deep parameter generation network model to form an end-to-end differentiable system. This allows the model to inherit the optimal linear state estimation characteristic of the Kalman filter while recursively outputting the covariance matrix of the state estimate and the covariance matrix of the observation prediction, thus providing the complete predicted distribution and confidence interval of future observations. This design fundamentally solves the inherent limitation of deep learning models, which can only output point estimates and cannot quantify prediction reliability, providing reliable prediction confidence information for high-risk decision-making scenarios such as financial risk control and medical diagnosis.

[0031] This application embeds deep learning into a structured state-space model framework, giving the model's internal parameters clear physical meanings: the state transition matrix reflects the dynamic evolution of the system, the observation matrix describes the mapping relationship between hidden states and observed variables, and the process noise covariance matrix and observation noise covariance matrix characterize the system's internal uncertainty and measurement error levels, respectively. By analyzing these parameter matrices, users can understand the system's dynamic patterns learned by the model, the driving factors of each variable, and the sources of uncertainty, effectively breaking the "black box" characteristic of deep learning and enhancing the model's acceptability in fields such as industrial control and scientific engineering.

[0032] This application couples a deep parameter generation network model with a Kalman filter module into a differentiable system. With the goal of maximizing prediction accuracy, the entire system is jointly optimized through backpropagation, enabling the deep parameter generation network to learn and generate the optimal time-varying parameter matrix. Simultaneously, this application supports online learning and adaptive mechanisms, allowing for incremental updates to the model as new data arrives, enabling the model to continuously adapt to changes in data distribution. This design combines the efficiency of offline training with online adaptive capabilities, avoiding the limitations of traditional deep learning models that are "fixed after training" and unable to adapt to environmental changes.

[0033] This application deeply integrates the nonlinear fitting and feature representation capabilities of deep learning with the structured state estimation and uncertainty measurement capabilities of linear quadratic Gaussian control, constructing a complete technical solution that simultaneously achieves nonlinear adaptive modeling, probabilistic prediction, interpretable analysis, and online learning within a single framework. This solution inherits the rigor and transparency of traditional control theory while leveraging the powerful pattern recognition capabilities of deep learning. It effectively solves the technical challenge of existing technologies where the linear assumptions of statistical models are limited, while deep learning models lack interpretability and uncertainty quantification capabilities. This provides an innovative solution for time series forecasting that combines accuracy, reliability, and interpretability. Attached Figure Description

[0034] To more clearly illustrate the technical solution of the present invention, the accompanying drawings used in the description will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0035] Figure 1 A flowchart illustrating the time series data prediction method provided in this application.

[0036] Figure 2 A flowchart illustrating another time series data prediction method provided in this application.

[0037] Figure 3 This is a flowchart illustrating an uncertainty quantification and cause identification method provided in this application.

[0038] Figure 4 A schematic diagram of the time series data prediction system provided in this application.

[0039] Figure 5 A schematic diagram of the structure of the electronic device provided in this application. Detailed Implementation

[0040] The various embodiments of this disclosure will be described more fully in the detailed steps of the time series data forecasting method described below. This disclosure may have various embodiments, and adjustments and changes may be made therein. However, it should be understood that there is no intention to limit the various embodiments of this disclosure to the specific embodiments disclosed herein, but rather this disclosure should be understood to cover all adjustments, equivalents, and / or alternatives falling within the spirit and scope of the various embodiments of this disclosure.

[0041] In the following, the terms “comprising” or “may include”, which may be used in various embodiments of this disclosure, indicate the presence of the disclosed functions, operations, or elements, and do not limit the addition of one or more functions, operations, or elements. Furthermore, as used in various embodiments of this disclosure, the terms “comprising,” “having,” and their cognates are intended only to indicate a particular feature, number, step, operation, element, component, or combination of the foregoing, and should not be construed as primarily excluding the presence of one or more other features, numbers, steps, operations, elements, components, or combinations of the foregoing, or the possibility of adding one or more combinations of the foregoing.

[0042] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0043] Please see Figure 1 The diagram shows a flowchart of a time series data prediction method in a specific embodiment. The method includes: S1: Preprocess the collected multi-source raw time-series data to generate a standardized training dataset, which includes historical time-series data blocks used for model training. In a specific implementation, the purpose of this step is to construct a high-quality, standardized training dataset from heterogeneous data sources.

[0044] Taking power load forecasting as an example, raw data is first collected from multiple sources, including core time-series data (such as load values ​​at 15-minute intervals), external covariate data (such as temperature, humidity, and holiday markers), and metadata (such as equipment status information). After collection, a unified timestamp operation is performed on all data to align data with different sampling frequencies to the same time granularity. For example, temperature data is interpolated from an hourly granularity to a 15-minute granularity to form an aligned time-series dataset.

[0045] Missing value detection and handling are performed on the aligned dataset. For randomly missing load values, linear interpolation is used for imputation; for consecutive missing load values ​​caused by holidays, forward imputation is used, with holiday markers input as independent features into the model. Next, outlier detection is performed using a sliding window Z-score method with a window size of 24 time points (corresponding to 6 hours). When the load value at a certain time point deviates from the window mean by more than 3 standard deviations, it is marked as an outlier. Detected outliers are corrected according to business logic, such as replacing them with the mean of adjacent normal values. Anomaly marker features are also added to the dataset to facilitate the model's learning of anomaly patterns.

[0046] The corrected dataset undergoes a stationarization transformation. Since the original load sequence typically exhibits clear diurnal periodicity and trends, a first-order differencing operation is performed to calculate load changes at adjacent time points, ensuring the sequence meets stationarity requirements. Subsequently, feature engineering is performed. For scenarios containing high-frequency sampled data, aggregated statistical features are calculated based on the original 15-minute interval load data as the model's observations. Set a time index. Indicates the first Each sampling period (e.g., one day), index of time intervals within the period. ,in The total number of intervals within the current period ( (corresponding to 96 15-minute intervals per day). For the first Within the first cycle The formula for calculating the aggregate statistical characteristics is: (The formula is missing from the original text.) This aggregated statistical feature is used as the observed value of the model. If other observed variables (such as external covariates) exist simultaneously, then a multidimensional observation vector is formed. .

[0047] Further calculations are performed on moving average features, relative strength index (RSI) features, and Bollinger Band width features as supplementary features. Interaction term features (such as the product of temperature and humidity) and lag term features (such as load values ​​lagged by 1 to 7 days) are also created between different variables. The aggregated statistical features, moving average features, RSI features, Bollinger Band width features, interaction term features, and lag term features generated by the above operations together constitute a feature sequence set. Z-score standardization is then performed independently on each feature sequence in the feature sequence set: Let the original value of a certain feature sequence be... Its mean is The standard deviation is ,implement Calculation, where and Computation is performed only on the training set to avoid data leakage. The standardized feature data is divided into multiple historical time series data blocks with consecutive time steps by a sliding partitioning operation with a fixed time window length (e.g., 20 time steps). Each historical time series data block contains standardized feature data for 20 consecutive time steps and is integrated into a normalized training dataset.

[0048] S2: Establish a linear Gaussian state-space model and define the key parameter matrix in the linear Gaussian state-space model as a time-varying parameter matrix to form a dynamic parameterized model framework.

[0049] In a specific implementation, this step transforms the traditional fixed state-space model into a data-driven dynamic parameterized model.

[0050] First, based on the training dataset output in step S1, a linear Gaussian state-space model is established. Taking electricity load forecasting as an example, let the state vector... Dimensions Used to capture potential dynamic patterns of load changes; observation vector Dimensions That is, the predicted load value; control input Dimensions It includes external covariates such as temperature, humidity, and holiday markers. The state equation and observation equation are defined as follows: Equations of state: Process noise ; Observation equation: Among them, observation noise .

[0051] in, For discrete-time indexing; for 3D state vector; for 3D observation vector; for The dimension-controlled input vector includes external covariates; for The state transition matrix; for The control input matrix; for The observation matrix; for A process noise vector of dimension , which follows a mean of zero and a covariance of . Gaussian distribution; for The observed noise vector has a mean of zero and a covariance of 1. Gaussian distribution; for The process noise covariance matrix; for The observation noise covariance matrix.

[0052] Unlike the traditional linear quadratic Gaussian control framework, this method uses all key parameter matrices , , , , Defined as time-varying, generated by data. Specifically, It reflects the internal dynamic evolution of the load system, such as the load inertia characteristics at different time periods; Characterize the degree of influence of external factors (such as temperature changes) on the system state; Describe the mapping relationship between hidden states and observed load values; This indicates inherent uncertainty in the system, such as random fluctuations in load forecasting that cannot be explained by the model. This represents the level of measurement error, such as noise introduced by the acquisition device. The structure and dimensions of these time-varying parameter matrices are explicitly defined, forming a dynamic parameterized model framework, which serves as the output target structure for subsequent deep parameter generation network models.

[0053] S3: Utilize deep parameter generation network model, based on historical time-series data blocks in the training dataset, and dynamically generate a set of time-varying parameter matrices for the current system state according to the dynamic parameterization model framework.

[0054] In a specific implementation, this step uses a specially designed deep parameter generation network model to map historical time-series data into dynamic system parameters.

[0055] First, historical time-series data blocks are extracted from the training dataset output in step S1. Each data block contains standardized feature data for 20 consecutive time steps, which serve as the input to the network model. A multi-layered, multi-path encoder-decoder deep parameter generation network model is constructed. Its output structure is determined by the dynamically parameterized model framework defined in step S2, namely the state dimension. Observation dimensions .

[0056] Historical time-series data blocks are input into the time-series feature encoder. In this embodiment, a Transformer encoder is used, which calculates the correlation strength between different time points through a self-attention mechanism, effectively capturing long-range dependencies in the load sequence and outputting a hidden state sequence rich in the temporal dynamics of the entire sequence. Pooling is performed on the hidden state sequence, and the final hidden state of the encoder or the sequence representation aggregated by the pooling layer is used as the context encoding vector of the entire input historical sequence. This vector is a highly condensed and abstracted representation of the load dynamics over the past 20 time steps.

[0057] The context-encoded vector is fed into a multi-branch parameter generation head, which has multiple parallel fully connected layer branches, each performing parameter generation operations on the corresponding matrix. Specifically, this includes: State transition matrix Generate a branch: Perform a fully connected mapping operation on the context encoding vector, outputting a vector that is then reshaped to form a vector of dimension . matrix To ensure the dynamic stability of the system, Constraints are imposed such that the magnitude of the eigenvalues ​​is less than 1, for example by spectral normalization or by forcing the eigenvalues ​​to be projected into the unit circle.

[0058] Observation matrix Generate a branch: Perform a fully connected mapping operation on the context encoding vector, generating a branch with dimension 1. matrix .

[0059] Noise covariance matrix and Generate a branch: Perform a fully connected mapping operation on the context encoding vector to generate its Cholesky factor, denoted as... for The Cholesky decomposition of the lower triangular matrix satisfies ,remember for The Cholesky decomposition of the lower triangular matrix satisfies Thus, a symmetric positive definite covariance matrix is ​​reconstructed.

[0060] Perform an integration operation on the parameter matrices output from each branch to form a set of dynamic system parameter matrices. By introducing a gating mechanism into the encoder, the network can automatically adjust the weights of the generation branches of each parameter based on the characteristics of the input sequence, thereby generating model parameters that are suitable for the current system state.

[0061] S4: Couple the depth parameter generation network model with the Kalman filter algorithm into an end-to-end differentiable system. Perform state estimation and prediction on the training dataset using the differentiable Kalman filter algorithm. Calculate the loss function based on the prediction error. Utilize the gradient of the loss function for backpropagation to optimize the depth parameter generation network model with the goal of maximizing prediction accuracy, thereby obtaining the optimized depth parameter generation network model. In this specific implementation, this step couples the depth parameter generation network model with the Kalman filter algorithm into an end-to-end differentiable system. First, the Kalman filter algorithm is transformed into a differentiable module, and the iterative formulas for the prediction step and the update step are defined.

[0062] The iterative formula for the prediction step includes: State prior estimation: ; Observational prior estimation: ; Prior estimation of covariance: .

[0063] The iterative formula for the update step includes: Kalman gain: ; Posterior state estimation: ; Posterior estimation of covariance: .

[0064] in, For discrete-time indexing; According to the date Time information Prior estimation of the state at time step; As of Time information Posterior estimation of the state at time step; According to the date Time information Prior estimates of time-based observations; Let be the covariance matrix of the prior state estimate; Let be the covariance matrix of the posterior state estimate; The Kalman gain matrix; It is an identity matrix with the same dimensions as the state vector; for The actual observed value at that moment.

[0065] Set the history window length Prediction window length Total time step Using the data from the first 20 time steps and the set of dynamic system parameter matrices generated in step S3. The Kalman filter update operation is performed step by step, initializing the Kalman filter state. Specifically, for the first time step, the Kalman filter state is estimated based on the initial state. Covariance Execute the prediction step to obtain and Then, using the actual observations from the first time step. Perform the update step to obtain and This process continues until the 20th time step, at which point the optimal state estimate based on the data from the previous 20 days is obtained.

[0066] Subsequently, after use Multi-step prediction and loss calculation operations are performed on data at each time step; the parameter matrix used in the prediction is uniformly set to... This is because the actual observations at future times are not yet available at the time of prediction, making it impossible to generate parameter matrices independently for each future time point. Based on the reasonable assumption that the system dynamics remain relatively stable in the short term, using the parameter matrix at the current time for multi-step prediction is a commonly used and effective strategy in engineering practice. For systems with strong non-stationarity, a rolling parameter generation strategy can be adopted.

[0067] For example, data from the next five time steps are then used for multi-step prediction and loss calculation. During the prediction phase, a parameter matrix is ​​used uniformly. Since the system state remains stable in the short term, using the same set of parameter matrices is reasonable. The loss function is then calculated, defined as the negative log-likelihood form:

[0068] in ,express The total uncertainty covariance matrix of the observations at time points; Representation matrix The determinant; This is the transpose of the prediction error vector; for The inverse matrix. The first term of the loss function prevents the model from... The second option involves setting the value too high to cheat. The weighted squared Mahalanobis distance measures the prediction error.

[0069] The gradient is calculated using the loss function, and the weights of the depth parameter generation network model are updated using the backpropagation algorithm. The optimized depth parameter generation network model is then output.

[0070] S5: The optimized depth parameter generation network model is used to dynamically generate a time-varying parameter matrix from the latest observation data, and multi-step forward prediction is performed based on the time-varying parameter matrix to output the probability prediction result.

[0071] In a specific implementation, this step is the application stage after the model training is completed. The network model is generated using the depth parameters optimized in step S4, and a forward propagation operation is performed based on the latest 20 time-series data blocks to dynamically generate a set of time-varying parameter matrices.

[0072] Multi-step forward prediction is performed based on the filtered state distribution, assuming the current time is... Predicting the future step, , For the preset maximum number of prediction steps (set to 5 in this embodiment), the following calculations are performed to generate the observation predictions and prediction covariance: State prediction: ; Covariance prediction: ; Predicted Observations: ; Predicting covariance: .

[0073] in, According to the date Time information Prediction of the state at any given moment; Here is the covariance matrix for state prediction; According to the date Time information Prediction of time-based observations; The covariance matrix of the predicted observations; , , , The parameter matrices for future time moments generated by the optimized deep parameter generation network model are uniformly set to values ​​since future observations are unknown during the prediction phase. That is, based on the current moment The parameter set is generated once from historical data, but to better reflect the temporal attributes of the parameters, this invention still uses... , , , This indicates that similar expressions will be used in subsequent sections and will not be repeated.

[0074] Based on the observed values ​​and predicted covariance, a prediction distribution construction operation is performed to generate future observed values. Predicted distribution Generate probability prediction results. Set the confidence level to [value missing]. ,in At the significance level, For standard normal distribution Quantiles. At a 95% confidence level. , Perform the confidence interval calculation operation to calculate the confidence interval for each observed variable:

[0075] in Indicates taking the matrix The vector consisting of the diagonal elements is squared to obtain the predicted standard deviation vector. This allows the prediction results to include not only point estimates but also quantified reliability information.

[0076] Simultaneously, interpretability analysis is performed: based on the time-varying parameter matrix set, the state transition matrix is ​​analyzed. Calculate its eigenvalues ​​and eigenvectors. If the magnitude of all eigenvalues ​​is less than 1, the system is stable. The eigenvectors reflect the main changing patterns of the load state, such as daily or weekly cycles. Analyze the observation matrix. To understand the mapping relationship between each observed variable (such as load value) and the hidden states, for example, which hidden states dominate the load changes. Analyze the process noise covariance matrix. and observation noise covariance matrix Compare the numerical values ​​of the two, if Significantly greater than This indicates that the uncertainty mainly originates from within the system (such as random load fluctuations); if Significantly greater than This indicates that the uncertainty mainly stems from the observation process (such as measurement noise). An interpretability analysis report is generated based on the analysis results.

[0077] Furthermore, this method supports online learning and adaptive mechanisms. When new observational data (such as the actual load value for the next day) arrives, the new data is input into step S1 for preprocessing, triggering step S4 to perform incremental updates on the deep parameter generation network model to adapt to changes in load distribution (such as seasonal changes or equipment aging), continuously optimizing model performance. For example, when new data accumulates to a preset threshold, the system selects data from the most recent W time steps using a sliding window approach, combines it with historical training data to perform incremental training, and employs elastic weight consolidation techniques to prevent catastrophic forgetting, thereby achieving continuous adaptive evolution of the model.

[0078] In this embodiment, by end-to-end coupling of a deep parameter generation network model with a differentiable Kalman filter algorithm, the deep network dynamically generates a time-varying parameter matrix based on historical time-series data to drive a linear quadratic Gaussian control framework. In the power load forecasting scenario, through a series of specific operations such as aggregated statistical feature calculation, dynamic generation of the time-varying parameter matrix, iterative update of the differentiable Kalman filter, construction of probability prediction distribution, and analysis of structured parameter matrix, the beneficial effects of simultaneously achieving adaptive modeling of complex nonlinear systems, quantification of prediction uncertainty, structured interpretability of the model decision process, and continuous online learning and adaptive capabilities are achieved within a single unified architecture. This effectively overcomes the technical bottlenecks of the linear assumption limitations of traditional statistical models and the lack of interpretability and uncertainty measurement in deep learning models, providing a complete technical solution for the field of time series forecasting that combines accuracy, reliability, and interpretability.

[0079] Furthermore, as a refinement and extension of the specific implementation of the above embodiments, another time series data prediction method is provided to fully illustrate the specific implementation process in this embodiment.

[0080] like Figure 2 As shown, the method includes the following steps: Step 1: Multi-source data acquisition and preprocessing.

[0081] This step is fundamental to the entire methodology and aims to build a high-quality, normalized training dataset from heterogeneous data sources.

[0082] Specifically, taking the prediction of the volatility of a financial asset as an example, the first step is to collect raw data from multiple sources, including core time-series data (such as high-frequency trading data), external covariate data (such as market indices and macroeconomic indicators), and relevant metadata (such as holiday markers). All data must be timestamped and aligned to the same time granularity to ensure time-series consistency.

[0083] As an example, when preprocessing the collected raw data, a time-series feature-based method can be used to handle missing values. For random missing values, linear interpolation or seasonally adjusted interpolation methods can be used, while for continuous missing values ​​(such as holidays), forward imputation combined with holiday markers as feature input can be used.

[0084] As an example, improved statistical methods can be used when detecting outliers, such as combining the Z-score method with a sliding window or the isolated forest algorithm, to correct or remove detected outliers according to business logic, and add outlier markers to the data.

[0085] As an example, for non-stationary sequences, the system performs differencing operations or relative rate of change calculations to ensure the stationarity of the sequence, such as converting price data into return data and volatility data into volatility change values.

[0086] As an example, advanced feature engineering can enhance raw data into features with higher information density and stronger representational capabilities. Taking financial products as an example, traditional daily volatility (such as the difference between the daily high and low prices) or volatility calculated based on closing prices provides limited information. Realized volatility is widely recognized by academia and industry as a superior volatility metric. It utilizes high-frequency intraday data (such as prices every 5 minutes) to more accurately and realistically capture the overall price fluctuations of an asset throughout the day, using this data as observations for the model. This can provide the model with higher quality and less noisy learning objectives. Specifically, the daily realized volatility can first be calculated based on high-frequency data. ,in Let N be the return of the i-th time interval within day t, and N be the total number of intervals within the trading session of that day. Then, classic technical indicators such as moving averages, relative strength index (RSI), and Bollinger Band width are calculated as supplementary features, and interaction terms and lag terms are created between different variables to capture complex nonlinear relationships. Finally, each feature sequence is independently Z-score standardized. ,in and Compute only on the training set, avoiding the use of information from the test set (future data).

[0087] Step 2: Dynamic system modeling and parameterization strategy.

[0088] The core innovation of this step is to transform the traditional fixed state-space model into a data-driven dynamic parameterized model, thus breaking through the linear assumption of the classic linear quadratic Gaussian (LQG) model and laying the foundation for subsequent adaptive learning of complex nonlinear dynamic characteristics.

[0089] Specifically, a linear Gaussian state-space model suitable for time series data prediction is first established, including the state equations. and observation equations , where the state vector The dimension n is determined based on data complexity and business requirements; the observation vector The dimension m is determined by the number of targets to be predicted, controlling the input. Includes all external covariates.

[0090] Furthermore, the fundamental difference between this invention and traditional LQG lies in the fact that all key parameter matrices are time-varying and generated by data-driven processes, wherein... Reflects the dynamic changes within the system. Characterize the degree of influence of external factors on the system state. Describe the mapping relationship from hidden states to observed values. This indicates the inherent uncertainty of the system. This indicates the measurement error level. For ease of explanation, the subscript 't' of the parameter matrix may be omitted in subsequent descriptions without causing confusion.

[0091] It is important to note that in some time series forecasting scenarios (such as financial market forecasting), although some external variables (such as interest rates, news sentiment, i.e., potential variables) can be observed, However, a common practice is not to treat them as independent control inputs. Instead, it is used as an observation value. Part of it. At this point, the system no longer has only volatility as an observation, but also an observation vector. It can contain multiple dimensions, for example: And hidden state The task then becomes capturing the common driving factors behind all observed variables (including the core predictive target and external covariates). Therefore, in the state transition matrix... During the learning process, the characteristics of external influences are learned and encoded in the state transition matrix. Therefore, in the following description, certain parts of the invention may be omitted. This does not affect the essential technology described in this invention.

[0092] Step 3: Design of depth parameter generation network.

[0093] This step is crucial for mapping raw time-series data into dynamic system parameters. Its goal is to automatically learn and generate all the key parameter matrices of the state-space model in the LQG control framework through a specially designed deep neural network, thereby achieving an adaptive transition from data-driven to model-driven control.

[0094] Specifically, the deep parameter generation network described in this invention is a multi-layered, multi-path encoder-decoder structure. The network input receives a standardized temporal data block after the first stage of preprocessing. This data block first enters a temporal feature encoder, which typically employs a bidirectional long short-term memory network (Bi-LSTM) or a Transformer encoder, depending on the domain of the data. Bi-LSTM can capture temporal dependencies in both forward and backward directions, associating the information at each time point with its past and future context, outputting a hidden state sequence rich in the temporal dynamics of the entire sequence. If a Transformer encoder is used, its self-attention mechanism is utilized to weight the correlation strength between different time points, thereby more effectively capturing long-range dependencies and key time nodes in the sequence.

[0095] In this network, the final hidden state of the encoder, or the sequence representation aggregated by the pooling layer, is considered as the context encoding vector of the entire input history sequence. This vector is a highly condensed and abstracted representation of the dynamic behavior over a past period. Specifically, this context encoding vector is fed into a multi-branch parameter generation head to achieve parameter dynamics. Unlike traditional single-output networks, the network described in this invention has multiple parallel fully connected layer branches, each responsible for generating parameters for a specific matrix in the state-space model, specifically including: 1. State transition matrix Generate a branch: This branch outputs a vector, which is then reshaped to form a vector of dimension . matrix To ensure the dynamic stability of the system, the generated... The matrix is ​​constrained, for example by using a specific activation function or post-processing technique to make the magnitude of its eigenvalues ​​less than 1, to prevent state divergence.

[0096] 2. Observation Matrix Generate a branch: This branch is generated. The matrix is ​​responsible for establishing a linear mapping relationship between the hidden state and the observed variables.

[0097] 3. Noise covariance matrix Branch generation: to ensure process noise covariance and observation noise covariance Due to the symmetric positive definiteness, the network does not directly generate these matrices, but rather generates their Cholesky decomposition factors (lower triangular matrices). and Then, matrix multiplication was performed. and This allows for the reconstruction of a positive definite covariance matrix that meets mathematical requirements. The above design not only guarantees the mathematical properties of the matrix but also makes the generation process differentiable, enabling optimization through gradient descent.

[0098] As an example, to enable networks to adapt to different system operating modes (such as stable and volatile periods in financial markets), gating mechanisms or adaptive attention can be introduced into the encoder. For instance, the network can automatically adjust the weights or biases of each parameter generation branch based on the characteristics of the current input sequence, thereby generating model parameters specifically suited to the current market state. Furthermore, techniques such as residual connections and layer normalization can be introduced to stabilize the training process of deep networks and prevent gradient vanishing or exploding.

[0099] The deep parameter generation network designed in this invention can be regarded as a special system identifier. It no longer relies on expert experience to manually set a fixed system model, but automatically infers the optimal model parameters that best describe the current and near-future system dynamics from data through end-to-end learning. This design deeply integrates the representation learning capabilities of deep learning with the modeling and description capabilities of control technology.

[0100] It is important to note that the data used in this step refers to the historical data processed in Step 1. For example, if the original data covers the period from January 1st to January 25th, 2025 (total number of days)... (sky), and let the historical window be The prediction window is Then this step only uses the previous one. Parameters are generated using data at each time step. .

[0101] Step 4: Differentiable filtering and end-to-end optimization.

[0102] This step transforms the optimal estimation algorithm in the control domain into a differentiable module, the purpose of which is to provide gradient signals for further optimization of the parameters obtained in step 3.

[0103] Specifically, the prediction step iterative formula is: State prior estimation Observational prior estimation Covariance prior estimation The update step iterative formula is: Kalman gain Post-state estimation Posterior estimation of covariance Innovative covariance matrix .

[0104] As an example, step 4 is mainly divided into two stages: (1) Before use The Kalman filter state is initialized with data for each time step. For example, taking the example in step 3, data from day 1 to day 20 and the parameters obtained in step 3 are used. The Kalman filter is updated daily and repeated until the 20th day, which yields the optimal state estimate based on the data from the previous 20 days.

[0105] (2) After use Multi-step prediction and loss calculation are performed using data at each time step. For example, taking the example in step 3, for day 21, the prior state for day 21 is first obtained using the prediction step (based on the state of day 20 and the control input for day 21, if any)). Then, the observation prior estimate (i.e., the predicted value for day 21) is calculated and compared with the actual value for day 21 to calculate the loss. Then, using the actual observation value for day 21, the state is updated through the update step to obtain the posterior state for day 21. The above steps are repeated in subsequent days to obtain the predicted values ​​for days 21 to 25.

[0106] It should be noted that the parameter matrix used in the prediction of the above stage (2) is uniformly set to... The main reason is that future observations are needed to generate parameter matrices for future time points. The method used in this invention is reasonable because market states do not change frequently in a very short period; for example, the dynamic characteristics of volatility in financial markets typically remain stable over a certain period.

[0107] In addition, in stages (1) and (2) above, not only are the states initialized with historical data, but the states are also updated with real values ​​during the prediction stage (this is called teacher forcing). This approach is common during training and can prevent error accumulation, making the training more stable.

[0108] Furthermore, in the above stages, the purpose of calculating the loss is to further update the parameter matrix through the loss. The loss function for calculating the loss is defined as follows:

[0109] in Composed of uncertainty in state prediction and observation noise, it represents the total uncertainty of the observation at time t; the first term in square brackets Used to prevent the model from Setting it too high can lead to cheating (because if...) If the first term approaches infinity, the second term will tend to 0, but the first term will tend to infinity; the second term inside the square brackets is... The squared weighted Mahalanobis distance measures the prediction error and also implies that if uncertainty... If the uncertainty is large, the penalty for the error will be smaller; if the uncertainty is small, the same error will be penalized more severely.

[0110] As an example, in terms of optimization strategy, this method can adopt a progressive training approach, starting with training from simple samples and gradually increasing the difficulty. It implements a multi-stage optimization strategy, first pre-training the encoder network and then fine-tuning the entire system end-to-end, and dynamically adjusting the learning rate scheduling strategy based on the validation set performance.

[0111] It is important to note that this step is not performing a traditional filtering algorithm, but rather constructing an intelligent filter that can be optimized and improved using derivatives (gradients).

[0112] Step 5: Probabilistic prediction reasoning and interpretability analysis.

[0113] This step is the application stage after the model training is completed. Its main purpose is to use the trained model to make predictions and provide an interpretation of the prediction results.

[0114] Specifically, in the prediction phase, multi-step forward prediction based on the model trained in the aforementioned steps and the filtered state distribution can yield single-point state predictions for the next k steps. Covariance prediction Observational prediction , and future observations Predicted distribution The parameter matrix at future time points , , , It is a temporary fixed parameter matrix dynamically generated based on the most recent historical window, obtained after training in step 4.

[0115] It's important to note that future inputs (external covariates) may be unknown. Therefore, in practice, there are typically two approaches: If the external covariates are known (e.g., time trends, holiday markers, etc.), future external covariates can be used directly to generate parameters; if the external covariates are unknown, they can be set to zero or replaced with predicted values, or a rolling forecast approach can be used (using the forecast from the previous time step as the input for the current time step) to generate parameters. The choice of approach depends on the specific application domain.

[0116] Furthermore, since the mean and covariance matrices of the predicted values ​​are obtained, their probabilistic predictions can be given. For example, for each observed variable, its 95% confidence interval can be calculated. , in, It is the quantile of the standard normal distribution (for a 95% confidence interval). ≈1.96), It is the predicted standard deviation vector.

[0117] As an example, the system dynamics learned by the model can be understood by analyzing the generated parameter matrix. First, the state transition matrix... It reflects the dynamic relationship between state variables. The magnitude and location of eigenvalues ​​can tell us about the stability of a system (the system is stable if the eigenvalues ​​are inside the unit circle). The eigenvectors represent the main change patterns of the state variables. Secondly, the observation matrix... This reflects how the hidden states are mapped to the observed variables. By analyzing which state variables primarily influence each observed variable, we can understand the driving factors of the observed variables. Additionally, such as... Figure 3 As shown, the noise covariance matrix and These represent the intensity of process noise and observation noise, respectively, and are compared. and The magnitude of the value can help us understand whether the model considers the uncertainty to originate primarily from within the system (process noise) or from the observation process (observation noise), thus enabling us to design optimization strategies.

[0118] As an example, one can determine the extent to which a feature affects the prediction by observing changes in the prediction results when the input feature is changed (e.g., by masking an external covariate).

[0119] As an example, by manually setting different initial state values ​​or changing the parameter matrix, the evolution of the system under different scenarios can be simulated, thereby understanding the behavior of the model.

[0120] In addition, this method also supports online learning and adaptive mechanisms. When new data arrives, the model can be incrementally updated using the training method in step 4 after preprocessing to adapt to changes in data distribution.

[0121] like Figure 4 As shown, the following are embodiments of the time series data prediction system provided in this disclosure. This system and the time series data prediction methods in the above embodiments belong to the same inventive concept. For details not described in detail in the embodiments of the time series data prediction system, please refer to the embodiments of the above time series data prediction methods.

[0122] A time series data prediction system, comprising: The data preprocessing module is used to preprocess the collected multi-source raw time-series data to generate a standardized training dataset, which includes historical time-series data blocks used for model training. The dynamic system model building module is used to build a linear Gaussian state-space model and define the key parameter matrix in the linear Gaussian state-space model as a time-varying parameter matrix, forming a dynamic parameterized model framework.

[0123] The deep parameter generation module is used to generate a network model using deep parameters. Based on the historical time-series data blocks in the training dataset, it dynamically generates a set of time-varying parameter matrices for the current system state according to the dynamic parameterization model framework. The set of time-varying parameter matrices includes a state transition matrix, an observation matrix, a process noise covariance matrix, and an observation noise covariance matrix. The differentiable filtering and optimization module is used to couple the depth parameter generation network model and the Kalman filter algorithm into an end-to-end differentiable system. It performs state estimation and prediction on the training dataset using the differentiable Kalman filter algorithm, calculates the loss function based on the prediction error, and uses the gradient of the loss function for backpropagation to optimize the depth parameter generation network model with the goal of maximizing prediction accuracy, thereby obtaining the optimized depth parameter generation network model. The prediction reasoning and interpretability analysis module is used to dynamically generate a time-varying parameter matrix from the latest observation data using the optimized deep parameter generation network model, and to perform multi-step forward prediction based on the time-varying parameter matrix, outputting the probability prediction result.

[0124] The time series data prediction system provided in this embodiment dynamically generates a time-varying parameter matrix by constructing a deep parameter generation network model and coupling it end-to-end with a differentiable Kalman filter algorithm. Within a unified framework, it realizes adaptive modeling of nonlinear systems, quantification of uncertainty in prediction results, interpretable analysis of the model decision process, and adaptive capability for continuous online learning. It effectively overcomes the technical bottlenecks of the linear assumption limitations of traditional statistical models and the lack of interpretability and uncertainty measurement in deep learning models.

[0125] Figure 5 A schematic diagram of the hardware structure of an electronic device for implementing various embodiments of the present invention.

[0126] The time series data prediction method provided in this application can be applied to electronic devices. Those skilled in the art will understand that the electronic device structure involved in the embodiments of this invention does not constitute a limitation on the electronic device. An electronic device may include more or fewer components than illustrated, or combine certain components, or have different component arrangements. In the embodiments of this invention, the electronic device includes, but is not limited to, laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the embodiments of this application described and / or claimed herein.

[0127] Electronic devices may include processors, external memory interfaces, internal memory, universal serial bus (USB) interfaces, charging management modules, power management modules, batteries, wireless communication modules, audio modules, speakers, microphones, sensor modules, buttons, cameras, displays, and SIM card interfaces, etc.

[0128] A processor may include one or more processing units, such as: a central processing unit (CPU), an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, memory, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural network processing unit (NPU). Different processing units may be independent devices or integrated into one or more processors.

[0129] The processor can serve as the nerve center and command center of an electronic device. The controller can generate operation control signals based on the instruction opcode and timing signals to control the fetching and execution of instructions.

[0130] The processor may also include memory for storing instructions and data. In some embodiments, the memory in the processor is a cache memory. This memory can store instructions or data that the processor has just used or that are used repeatedly. If the processor needs to use the instruction or data again, it can retrieve it directly from this memory. This avoids repeated accesses, reduces processor latency, and thus improves system efficiency.

[0131] An external storage interface (ESI) can be used to connect external memory cards, such as microSD cards, to expand the storage capacity of electronic devices. The external memory card communicates with the processor through the ESI to perform data storage functions, such as saving music and video files on the external memory card.

[0132] Internal memory can be used to store computer executable program code, which includes instructions. The processor executes various functional applications and data processing of electronic devices by running the instructions stored in internal memory. Internal memory can include a program storage area and a data storage area. Internal memory can include high-speed random access memory, and can also include non-volatile memory, such as at least one disk storage device, flash memory device, universal flash storage (UFS), etc.

[0133] Wireless communication functionality in electronic devices can be achieved through antennas, wireless communication modules, modem processors, and baseband processors.

[0134] Wireless communication modules can provide solutions for wireless communication applications in electronic devices, including wireless local area networks (WLANs) (such as wireless fidelity (Wi-Fi) networks), Bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), and infrared (IR) technologies.

[0135] Electronic devices can implement audio functions through audio modules, speakers, receivers, microphones, headphone jacks, and application processors.

[0136] Electronic devices can achieve shooting functions through ISPs, cameras, video codecs, GPUs, displays, and application processors.

[0137] Electronic devices can achieve display functions through GPUs, displays, and application processors.

[0138] A GPU is a microprocessor for image processing, connected to the display screen and application processor. GPUs perform mathematical and geometric calculations for graphics rendering. A processor may include one or more GPUs, which execute program instructions to generate or modify display information.

[0139] A display screen is used to display images, videos, etc. A display screen includes a display panel.

[0140] The aforementioned electronic device realizes the time series data prediction method of this application by coupling the deep parameter generation network model with the differentiable Kalman filter algorithm end-to-end. It utilizes the deep network to dynamically generate time-varying parameter matrices based on historical time series data to drive a linear quadratic Gaussian control framework. This achieves the beneficial effects of simultaneously realizing adaptive modeling of complex nonlinear systems, quantification of prediction uncertainty, structured and interpretable model decision-making process, and continuous online learning and adaptive capabilities within a single unified architecture.

[0141] The storage medium provided in this application stores a program product capable of implementing a time series data prediction method.

[0142] Time series data prediction methods include: The collected multi-source raw time-series data are preprocessed to generate a standardized training dataset, which includes historical time-series data blocks used for model training. A linear Gaussian state-space model is established, and the key parameter matrix in the linear Gaussian state-space model is defined as a time-varying parameter matrix, forming a dynamic parameterized model framework. A deep parameter generation network model is used to dynamically generate a set of time-varying parameter matrices for the current system state based on historical time-series data blocks in the training dataset and in accordance with a dynamic parameterization model framework. The set of time-varying parameter matrices includes a state transition matrix, an observation matrix, a process noise covariance matrix, and an observation noise covariance matrix. The deep parameter generation network model is coupled with the Kalman filter algorithm into an end-to-end differentiable system. The differentiable Kalman filter algorithm is executed on the training dataset for state estimation and prediction. The loss function is calculated based on the prediction error. The gradient of the loss function is backpropagated to optimize the deep parameter generation network model with the goal of maximizing prediction accuracy, resulting in the optimized deep parameter generation network model. The optimized depth parameter generation network model is used to dynamically generate a time-varying parameter matrix from the latest observation data, and multi-step forward prediction is performed based on the time-varying parameter matrix to output the probability prediction result. In some possible implementations, the time series data prediction method of this disclosure can be implemented as a program product including program code that, when the program product is run on a terminal device, causes the terminal device to perform the steps described in the "Exemplary Methods" section of this specification according to various exemplary embodiments of this disclosure.

[0143] The storage medium disclosed herein may be any combination of one or more readable media. A readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples (a non-exhaustive list) of readable storage media include: an electrical connection having one or more wires, a portable disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof.

[0144] The above description of the disclosed embodiments enables those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for predicting time series data, characterized in that, include: The collected multi-source raw time-series data are preprocessed to generate a standardized training dataset, which includes historical time-series data blocks used for model training. A linear Gaussian state-space model is established, and the key parameter matrix in the linear Gaussian state-space model is defined as a time-varying parameter matrix, forming a dynamic parameterized model framework. A deep parameter generation network model is used to dynamically generate a set of time-varying parameter matrices for the current system state based on historical time-series data blocks in the training dataset and in accordance with a dynamic parameterization model framework. The set of time-varying parameter matrices includes a state transition matrix, an observation matrix, a process noise covariance matrix, and an observation noise covariance matrix. The deep parameter generation network model is coupled with the Kalman filter algorithm into an end-to-end differentiable system. The differentiable Kalman filter algorithm is executed on the training dataset for state estimation and prediction. The loss function is calculated based on the prediction error. The gradient of the loss function is backpropagated to optimize the deep parameter generation network model with the goal of maximizing prediction accuracy, resulting in the optimized deep parameter generation network model. The optimized depth parameter generation network model is used to dynamically generate a time-varying parameter matrix from the latest observation data, and multi-step forward prediction is performed based on the time-varying parameter matrix to output the probability prediction result.

2. The time series data prediction method according to claim 1, characterized in that, The preprocessing of the collected multi-source raw time-series data to generate a standardized training dataset includes: Receive raw data collected from multiple channels, including core time-series data, external covariate data, and metadata; perform a unified timestamp operation on the raw data and align all data to the same time granularity to generate an aligned time-series dataset; Missing value detection is performed on the aligned time series dataset, and missing values ​​are processed using a time series feature-based method. After missing value imputation, a complete dataset is generated. Outlier detection is performed on the complete dataset, using a Z-score method combined with a sliding window or an isolated forest algorithm to identify outliers; the detected outliers are corrected or removed, and anomaly labels are generated and added to the complete dataset to generate a corrected dataset. Perform differencing operations or relative rate of change calculations on the non-stationary sequences in the corrected dataset to ensure the stationarity of the sequences and generate a stationary dataset; Perform advanced feature engineering operations on the stabilized dataset, specifically including: For scenarios containing high-frequency sampled data, aggregated statistical features are calculated based on the high-frequency data and used as the observations of the model. Set a time index Indicates the first Each sampling period, index of time intervals within the period ,in This represents the total number of intervals within the current period. For the first Within the first cycle The rate of change of the variable over time intervals; using the formula Calculate aggregate statistical features; Calculate moving average features, relative strength index features, and Bollinger Band width features as supplementary features, and create interaction term features and lag term features between different variables; A feature sequence set is constructed based on aggregate statistical features, moving average features, relative strength index features, Bollinger Band width features, interaction term features, and lag term features. Z-score normalization is performed independently on each feature sequence in the feature sequence set. The normalized feature data is then divided into sliding segments according to a fixed time window length to construct multiple historical time series data blocks with consecutive time steps. Each historical time series data block contains normalized feature data with multiple consecutive time steps. Historical time-series data blocks are integrated into a normalized training dataset.

3. The time series data prediction method according to claim 2, characterized in that, The process involves establishing a linear Gaussian state-space model and defining the key parameter matrices in the linear Gaussian state-space model as time-varying parameter matrices, forming a dynamic parameterized model framework, including: Based on the training dataset, a linear Gaussian state-space model suitable for time series data prediction is established, including the state equation and the observation equation: The state equation is Its process noise ; The observation equation is Its observation noise ; in, For discrete-time indexing; for dimensional state vector, for 3D observation vector, for The dimension-controlled input vector includes external covariates; for The state transition matrix; for The control input matrix; for The observation matrix; for A process noise vector of dimension , which follows a mean of zero and a covariance of . Gaussian distribution; for The observed noise vector has a mean of zero and a covariance of 1. Gaussian distribution; for The process noise covariance matrix; for The observation noise covariance matrix; For all key parameter matrices in the linear Gaussian state-space model structure , , , , The time-varying definition operation is executed, setting it to be time-varying and data-driven, generating a time-varying parameter matrix definition; where, Used to reflect the dynamic changes within the system. Used to characterize the degree of influence of external factors on the system state. Used to describe the mapping relationship from hidden states to observed values. Used to represent the inherent uncertainty of a system. Used to represent the measurement error level; outputs the definition of a time-varying parameter matrix; The structure and dimension definition operations are performed on the time-varying parameter matrix to form a dynamic parameterized model framework.

4. The time series data prediction method according to claim 3, characterized in that, The deep parameter generation network model dynamically generates a set of time-varying parameter matrices for the current system state based on historical time-series data blocks in the training dataset, following a dynamic parameterization model framework. This includes: Extract historical time-series data blocks from the training dataset; A multi-level, multi-path encoder-decoder structure deep parameter generation network model is constructed, and the output structure of the deep parameter generation network model is determined by a dynamic parameterized model framework. Historical time series data blocks are input into the depth parameters to generate a time series feature encoder for the network model, which performs time series dependency capture operations in both forward and backward directions to generate a hidden state sequence; the time series feature encoder adopts a bidirectional long short-term memory network or a Transformer encoder. Pooling is performed on the hidden state sequence, and the final hidden state of the temporal feature encoder or the sequence representation aggregated by the pooling layer is used as the context encoding vector. The context encoding vector is input into the multi-branch parameter generation head of the deep parameter generation network model. The multi-branch parameter generation head has multiple parallel fully connected layer branches. The parameter generation operation of the corresponding matrix is ​​performed through each fully connected layer branch to generate the parameter matrix. The fully connected layer branches include: State transition matrix The generation branch is used to perform a fully connected mapping operation on the context encoding vector, outputting a vector that is reshaped to a dimension of . matrix ; Observation matrix The generation branch is used to perform a fully connected mapping operation on the context encoding vector, and the generation dimension is... matrix ; Noise covariance matrix and The generation branch is used to perform a fully connected mapping operation on the context encoding vector to generate its Cholesky factor, denoted as... for The Cholesky decomposition of the lower triangular matrix satisfies ,remember for The Cholesky decomposition of the lower triangular matrix satisfies ; Perform an integration operation on the parameter matrix to form a set of dynamic system parameter matrices. .

5. The time series data prediction method according to claim 4, characterized in that, The process involves coupling the depth parameter generation network model with the Kalman filter algorithm into an end-to-end differentiable system. A differentiable Kalman filter algorithm is executed on the training dataset for state estimation and prediction. A loss function is calculated based on the prediction error. The gradient of the loss function is used for backpropagation to optimize the depth parameter generation network model with the goal of maximizing prediction accuracy, resulting in an optimized depth parameter generation network model. This includes: Based on the set of dynamic system parameter matrices, the Kalman filter algorithm is transformed into a differentiable Kalman filter module, and the iterative formulas for the prediction step and the update step are defined. The iterative formula for the prediction step includes: in, For discrete-time indexing; According to the date Time information Prior estimation of the state at time step; As of Time information Posterior estimation of the state at time step; According to the date Time information Prior estimates of time-based observations; Let be the covariance matrix of the prior state estimate; The iterative formula for the update step includes: in, Here is the covariance matrix of the posterior state estimate. Here is the Kalman gain matrix. It is an identity matrix with the same dimensions as the state vector; for The actual observed value at time; Let the length of the history window be... The prediction window length is Total time step Before use A set of data and dynamic system parameter matrices at each time step The Kalman filter update operation is performed step by step using the differentiable Kalman filter module to initialize the Kalman filter state and generate the initial state estimate. After use Multi-step prediction and loss calculation operations are performed on data at each time step; the parameter matrix used in the prediction is uniformly set to... Using the loss function Calculate the loss function value; ,express The total uncertainty covariance matrix of the observations at time points; Representation matrix The determinant of; This is the transpose of the prediction error vector; for The inverse matrix; Perform gradient calculation on the loss function value, and calculate the gradient of the loss function with respect to the weights of the depth parameter generation network model using the backpropagation algorithm; use the calculated gradient to perform weight update operation, and update the weights of the depth parameter generation network model; The updated weights are loaded into the deep parameter generation network model to generate an optimized deep parameter generation network model.

6. The time series data prediction method according to claim 5, characterized in that, The process involves using the optimized depth parameter generation network model to dynamically generate a time-varying parameter matrix from the latest observation data, and performing multi-step forward prediction based on the time-varying parameter matrix to output probabilistic prediction results, including: The optimized depth parameters are used to generate a network model. Forward propagation is performed based on historical time-series data blocks of the latest observation data to dynamically generate a set of time-varying parameter matrices. A multi-step forward prediction operation is performed based on the filtered state distribution. Let the current time be... Predicting the future step, , To generate the observation predictions and prediction covariance by performing the following calculations, based on the preset maximum number of prediction steps: in, According to the date Time information Prediction of the state at any given moment; Here is the covariance matrix for state prediction; According to the date Time information Prediction of time-based observations; The covariance matrix of the predicted observations; , , , The future time-time parameter matrix is ​​generated by the optimized depth parameter generation network model; Based on the observed values ​​and predicted covariance, a prediction distribution construction operation is performed to provide future observed values. Predicted distribution Generate probability prediction results; Set the credit level to ,in At the significance level, For standard normal distribution Quantiles; based on probability prediction results, confidence interval calculation is performed using the formula... Calculate the confidence interval for each observed variable; where, Indicates taking the matrix A vector consisting of the diagonal elements; Based on the time-varying parameter matrix set, the state transition matrix is ​​analyzed to understand the system's stability and main change patterns through its eigenvalues ​​and eigenvectors; the observation matrix is ​​analyzed to understand the mapping relationship between each observed variable and the hidden state; the process noise covariance matrix and the observation noise covariance matrix are analyzed, and their numerical values ​​are compared to determine whether the uncertainty mainly originates from within the system or from the observation process. An interpretability analysis report is generated based on the analysis results.

7. The time series data prediction method according to claim 4, characterized in that, The state transition matrix Branch generation is achieved by generating branches. The matrix is ​​constrained so that the modulus of its eigenvalues ​​is less than 1 to ensure the dynamic stability of the system; a gating mechanism or adaptive attention is introduced into the temporal feature encoder of the deep parameter generation network model to automatically adjust the weights or biases of each parameter generation branch according to the features of the current input sequence.

8. A time series data prediction system, characterized in that, The system employs the time series data prediction method as described in any one of claims 1 to 7; The system includes: The data preprocessing module is used to preprocess the collected multi-source raw time-series data to generate a standardized training dataset, which includes historical time-series data blocks used for model training. The dynamic system model building module is used to build a linear Gaussian state-space model and define the key parameter matrix in the linear Gaussian state-space model as a time-varying parameter matrix, forming a dynamic parameterized model framework. The deep parameter generation module is used to generate a network model using deep parameters. Based on the historical time-series data blocks in the training dataset, it dynamically generates a set of time-varying parameter matrices for the current system state according to the dynamic parameterization model framework. The set of time-varying parameter matrices includes a state transition matrix, an observation matrix, a process noise covariance matrix, and an observation noise covariance matrix. The differentiable filtering and optimization module is used to couple the depth parameter generation network model and the Kalman filter algorithm into an end-to-end differentiable system. It performs state estimation and prediction on the training dataset, calculates the loss function based on the prediction error, and uses the gradient of the loss function for backpropagation to optimize the depth parameter generation network model with the goal of maximizing prediction accuracy, thereby obtaining the optimized depth parameter generation network model. The prediction reasoning and interpretability analysis module is used to dynamically generate a time-varying parameter matrix from the latest observation data using the optimized deep parameter generation network model, and to perform multi-step forward prediction based on the time-varying parameter matrix, outputting the probability prediction result.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the steps of the time series data prediction method as described in any one of claims 1 to 7.

10. A storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the time series data prediction method as described in any one of claims 1 to 7.

Images