A multi-element time series prediction method and device

Abstract

This application discloses a multivariate time series prediction method and apparatus, relating to the field of data prediction technology. The method includes: constructing a network model based on a graph convolutional neural network according to the topology of a target nonlinear dynamic system; the network model includes an adaptive graph module, an encoder, a spatial neural differential equation module, a temporal neural differential equation module, and a decoder connected in sequence; training the network model using a historical dataset of the target nonlinear dynamic system to obtain a multivariate time series prediction model; introducing state feedback through the spatial neural differential equation module to reveal the evolution pattern of the spatiotemporal time series in the spatial dimension, and introducing neural differential equations based on nonlinear state transition theory to simulate the state evolution at the temporal level, thereby improving the effectiveness of data prediction by fusing spatial features while suppressing feature oversmoothing.

Description

Technical Field

[0001] This application relates to the field of data prediction technology, and in particular to a multivariate time series prediction method and apparatus. Background Technology

[0002] Spatiotemporal time series are widely present in current production and daily life, and modeling them can provide a theoretical basis for decision-making. The evolution of many phenomena in the real world can be modeled as nonlinear dynamic systems involving the interaction of multiple objects and their longitudinal evolution over time, such as traffic flow and temperature changes. Multivariate time series are nonlinear data with dynamic characteristics generated by nonlinear dynamic systems, which can characterize the system's operating state. Modeling and predicting spatiotemporal time series can reveal the evolutionary laws of systems, explore their future states, and provide a theoretical basis for the regulation and decision-making of complex systems. For example, predicting electricity generation can help governments better allocate power resources. Simultaneously studying air quality at multiple locations helps countries macro-control pollutant emissions. Accurate traffic flow prediction can help people improve travel efficiency and reduce commuting time.

[0003] To fully extract the potential information from spatiotemporal time series, it is necessary to model both spatial topology and temporal dependencies. Since traditional Convolutional Neural Networks (CNNs) cannot maintain spatial translation invariance on graph structures, researchers commonly employ Graph Neural Networks (GNNs) to model spatial features in multivariate time series. For GNN-based models, researchers typically use adjacency matrices to model the connections between variables (also known as nodes). However, while deeper GNN models offer stronger representational capabilities, they can also lead to oversmoothing of node features, causing the feature vectors of each node to become too similar, resulting in significant input redundancy and weakening prediction performance. Summary of the Invention

[0004] The purpose of this application is to provide a multivariate time series forecasting method and apparatus that can improve the effectiveness of data forecasting.

[0005] To achieve the above objectives, this application provides the following solution:

[0006] Firstly, this application provides a multivariate time series forecasting method, the multivariate time series forecasting method comprising:

[0007] Based on the topology of the target nonlinear dynamic system, a network model based on a graph convolutional neural network is constructed. The network model includes an adaptive graph module, an encoder, a spatial neural differential equation module, a temporal neural differential equation module, and a decoder connected in sequence. The adaptive graph module is determined based on the topology of the target nonlinear dynamic system.

[0008] The network model is trained using a historical dataset of the target nonlinear dynamical system to obtain a multivariate time series prediction model; each sample in the historical dataset is a multivariate time series; the adaptive graph module is used to compress the multidimensional input data and provide an adjacency matrix for the encoder; the encoder is used to extract nonlinear spatiotemporal features from the multidimensional input data based on the adjacency matrix, providing initial values ​​for spatial evolution for the spatial neural differential equation module; the spatial neural differential equation module, based on the initial values ​​of spatial evolution, combines a state feedback mechanism with graph ordinary differential equations to fuse spatial features while suppressing feature oversmoothing, providing initial values ​​for temporal evolution for the temporal neural differential equation module; the temporal neural differential equation module, based on the initial values ​​of temporal evolution, obtains the multidimensional state variables corresponding to the next time step of the multidimensional input data through state transition; the decoder is used to decode the multidimensional state variables to obtain predicted values.

[0009] The multivariate time series prediction model is used to predict the multivariate time data of the next time step based on the multivariate time data of the previous time step.

[0010] Optionally, each node in the adaptive graph module is assigned an adaptive embedding vector, which is used to represent the low-dimensional data after compression of the multi-dimensional input data.

[0011] Optionally, the adjacency matrix is ​​represented as: ;

[0012] in, Represents the adjacency matrix. , For the first i The node and the first j The connection weights of edges between nodes, where T represents the transpose. Indicates the first i The adaptive embedding vector of each node. N The number of nodes.

[0013] Optionally, the encoder includes a first linear layer and a graph convolutional neural network.

[0014] Optionally, the initial value of the spatial layer evolution is represented as:

[0015] ;

[0016] in, The first expression of the spatial neural differential equation k Step evolution state, for Dimensional data, For multidimensional input data, for Dimensional data, N Let h be the number of nodes in the adaptive graph module, h be the step size, and D be the feature dimension. For the first k Learnable parameters of a layered graph convolutional neural network.

[0017] Optionally, based on the initial values ​​of spatial evolution, by combining a state feedback mechanism with graph ordinary differential equations, spatial features are fused while suppressing feature oversmoothing, providing initial values ​​of temporal evolution for the temporal neural differential equation module, specifically including:

[0018] The Euler trapezoidal integral method is used to solve the forward propagation equation of the graph convolutional neural network based on nonlinear state feedback, and the evolution state after k propagations is obtained.

[0019] The first k The evolutionary state after the second propagation serves as the initial value for the evolution at the temporal level.

[0020] The forward propagation equation of the graph convolutional neural network based on nonlinear state feedback is expressed as: ;

[0021] in, Indicates the first k The state of a layered graph convolutional neural network. Represents the adjacency matrix. The first expression of the spatial neural differential equation k Step evolution state, Represents the feedback function. and These are the reference input control matrix and the reference output control matrix of the target nonlinear dynamic system, respectively. This is the input control matrix.

[0022] Optionally, based on the initial values ​​of time-level evolution, the multidimensional state variables corresponding to the multidimensional input data at the next time step are obtained through state transition, specifically including:

[0023] The mechanistic evolution equation of the time-level evolution is solved by the Euler trapezoidal integral method to obtain the multidimensional state variables corresponding to the multidimensional input data at the next time step;

[0024] The multidimensional state quantity at the next moment is represented as follows:

[0025] ;

[0026] Where T represents transpose. for Multidimensional state variables at time. for Multidimensional state variables at time. for Multidimensional state variables at time. t For time, To predict the time step, It is a sandwich network structure, which includes a second linear layer, a long short-term memory network, and a third linear layer connected in sequence.

[0027] Optionally, the loss function for training the network model is expressed as: ;

[0028] in, L This is the total loss value. For mean square error loss, To mitigate losses in the stability criterion section;

[0029] ;

[0030] in, Represents system functions. The state at time step zero. For ReLU activation functions, Energy function The derivative of , Indicates the first n The characteristics of each node N is the number of nodes in the adaptive graph module.

[0031] Optionally, the target nonlinear dynamic system is a traffic system, which includes multiple monitoring nodes. The multivariate time series prediction model is used to predict the traffic flow of each monitoring node at the next moment based on the traffic flow of each monitoring node at the previous moment.

[0032] Secondly, this application provides a multivariate time series forecasting device, the multivariate time series forecasting device comprising:

[0033] A network model construction module is used to construct a network model based on a graph convolutional neural network according to the topology of the target nonlinear dynamic system. The network model includes an adaptive graph module, an encoder, a spatial neural differential equation module, a temporal neural differential equation module, and a decoder connected in sequence. The adaptive graph module is determined based on the topology of the target nonlinear dynamic system.

[0034] A network model training module is used to train the network model using a historical dataset of the target nonlinear dynamic system to obtain a multivariate time series prediction model; each sample in the historical dataset is a multivariate time series. An adaptive graph module is used to compress multidimensional input data and provide an adjacency matrix for the encoder. The encoder is used to extract nonlinear spatiotemporal features from the multidimensional input data based on the adjacency matrix, providing initial values ​​for spatial evolution to the spatial neural differential equation module. The spatial neural differential equation module, based on the initial values ​​of spatial evolution, combines a state feedback mechanism with graph ordinary differential equations to fuse spatial features while suppressing feature oversmoothing, providing initial values ​​for temporal evolution to the temporal neural differential equation module. The temporal neural differential equation module, based on the initial values ​​of temporal evolution, obtains the multidimensional state variables corresponding to the next time step of the multidimensional input data through state transitions. The decoder is used to decode the multidimensional state variables to obtain predicted values.

[0035] The prediction module is used to predict the multivariate time data of the next time step based on the multivariate time data of the previous time step using the multivariate time series prediction model.

[0036] According to the specific embodiments provided in this application, the following technical effects are disclosed: This application provides a multivariate time series prediction method and apparatus, which introduces state feedback through a spatial neural differential equation module to reveal the evolution pattern of spatiotemporal time series in the spatial dimension, and introduces neural differential equations based on nonlinear state transition theory to simulate the state evolution at the time level. While fusing spatial features, it suppresses the phenomenon of feature oversmoothing, thereby improving the effectiveness of data prediction. Attached Figure Description

[0037] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0038] Figure 1 This is a flowchart illustrating a multivariate time series prediction method provided in an embodiment of this application.

[0039] Figure 2 This is a schematic diagram illustrating the principle of a multivariate time series prediction method provided in an embodiment of this application.

[0040] Figure 3 This is a schematic diagram of the structure of a multivariate time series prediction model provided in an embodiment of this application. Detailed Implementation

[0041] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0042] To make the above-mentioned objectives, features and advantages of this application more apparent and understandable, the application will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0043] Spatiotemporal time series can generally be viewed as being generated by complex nonlinear dynamic systems, exhibiting strong temporal dependence and spatial correlation. Related prediction methods typically rely on fitting training data and labels, leading to poor model interpretability. To address this, this application proposes a graph differential equations based on state feedback (SF-GDE) for multivariate time series prediction with a degree of interpretability. This application introduces state feedback into the adaptive graph module to reveal the spatial evolution patterns of spatiotemporal time series and introduces neural differential equations (NDEs) based on nonlinear state transition theory to simulate temporal state evolution. State feedback, starting from the idea of ​​mechanistic modeling, directly acts on the system matrix, adjusting the output of the graph convolutional network to solve the oversmoothing phenomenon of deep graph convolutional networks (GCNs). Specifically, this application provides a multivariate time series prediction method, such as... Figure 1 As shown, the multivariate time series prediction method includes steps 101 to 103.

[0044] Step 101: Based on the topology of the target nonlinear dynamic system, construct a network model based on a graph convolutional neural network; the network model includes an adaptive graph module, an encoder, a spatial neural differential equation module, a temporal neural differential equation module, and a decoder connected in sequence; the adaptive graph module is determined based on the topology of the target nonlinear dynamic system.

[0045] Step 102: Train the network model using the historical dataset of the target nonlinear dynamic system to obtain a multivariate time series prediction model; each sample in the historical dataset is a multivariate time series; the adaptive graph module is used to compress the multidimensional input data and provide an adjacency matrix for the encoder; the encoder is used to extract nonlinear spatiotemporal features from the multidimensional input data based on the adjacency matrix and provide initial values ​​for spatial evolution for the spatial neural differential equation module; the spatial neural differential equation module is used to provide initial values ​​for temporal evolution for the temporal neural differential equation module by combining a state feedback mechanism and graph ordinary differential equations based on the initial values ​​of spatial evolution, fusing spatial features while suppressing feature oversmoothing; the temporal neural differential equation module is used to obtain the multidimensional state quantity corresponding to the next time step of the multidimensional input data through state transition based on the initial values ​​of temporal evolution; the decoder is used to decode the multidimensional state quantity to obtain the predicted value.

[0046] Step 103: Use the multivariate time series prediction model to predict the multivariate time data of the next time step based on the multivariate time data of the previous time step.

[0047] This application introduces state feedback through a spatial neural differential equation module to reveal the evolution pattern of spatiotemporal time series in the spatial dimension, and introduces neural differential equations based on nonlinear state transition theory to simulate the state evolution at the time level. While integrating spatial features, it suppresses the phenomenon of feature oversmoothing, thereby improving the effectiveness of data prediction.

[0048] In one exemplary embodiment, the multivariate time series forecasting method of this application is specifically a state feedback-based graph differential equation multivariate time series forecasting method, such as... Figure 2 As shown, this includes: First, setting a learnable node embedding vector for each node in a data-driven manner. The network topology is adaptively learned by calculating the cosine similarity between the embedding vectors of different nodes. Considering the strong initial value sensitivity of NDEs, in order to make spatial NDEs more accurately simulate information propagation on the graph, the original input is transformed through linear mapping and graph convolution operations. Mapping to a higher-dimensional space yields the initial evolutionary state. and Then, by continuously solving the spatial NDE, the accurate initial evolution state of the temporal NDE is obtained. and Finally, the dynamic evolution mechanism of the time-based NDE simulation system is used to infer the system's future... higher-dimensional state of the step The final prediction is obtained by decoding the data through a decoder. .

[0049] This application first studies spatial topology. With the development of sensor technology, multivariate time series are gradually exhibiting large-scale and high-dimensional characteristics. To overcome the limitations of input data scale, an adaptive graph algorithm is proposed to explicitly model the implicit topological structure in multivariate time series. Each node in the adaptive graph module is assigned an adaptive embedding vector, which represents the low-dimensional data after compression of the multivariate input data.

[0050] Adaptive Embedding Vector ,in d It is a manually given hyperparameter that represents the embedding dimension of a node. Data-driven, it adaptively learns the potential relationships between nodes in a data-driven manner. During training, the high-dimensional input of the original nodes... In low dimension, it is represented as Therefore, compared to directly inputting the raw data... Learning Topology N Indicates the number of nodes. h Compared to other methods (indicating time step size), this adaptive graph algorithm can effectively mine network topology in large-scale time series data while reducing computational complexity. Furthermore, SF-GDE obtains adaptive node embedding vectors that can characterize the original nodes. Then, the cosine similarity between each node is calculated to measure whether there is a connecting edge between two nodes and the weight of a given edge.

[0051] The adjacency matrix is ​​represented as: (1)

[0052] in, Represents an adjacency matrix (also known as a graph structure). , For the first i The node and the first j The connection weights of edges between nodes, where T represents the transpose. Indicates the first i The adaptive embedding vector of each node. N Let A be the number of nodes. SF-GDE considers the evolution of the nodes themselves, therefore self-loops are added to the adjacency matrix A, i.e. At this point, the interaction relationships between nodes are obtained and fed into the step of learning the initial spatial representation to guide the fusion of spatial features between nodes. The adjacency matrix can reduce the computational complexity of the algorithm while breaking the limitation of input data length.

[0053] The encoder includes a first linear layer and a graph convolutional neural network.

[0054] Considering the strong initial value sensitivity of NDEs, this step aims to provide the spatial NDE with an initial value that can more accurately represent the initial state of the space. Specifically, SF-GDE obtains the initial evolution state of the spatial NDE by mapping the original input data X to a high-dimensional space. By mapping the original input to a high-dimensional space, the nonlinear features in the multivariate time series can be linearized, making the spatial NDE easier and more accurate to simulate the information propagation process on the graph. The specific mapping equation is as follows:

[0055] (2)

[0056] in, The first expression of the spatial neural differential equation k Step evolution state, for Dimensional data, For multidimensional input data, for Dimensional data, N Let h be the number of nodes, h be the step size, and D be the feature dimension. For the graph convolutional neural network, the first... k Learnable parameters of the layer.

[0057] In an exemplary embodiment, based on the initial value of spatial evolution, by combining a state feedback mechanism with graph ordinary differential equations, spatial features are fused while suppressing feature oversmoothing, providing the initial value of temporal evolution for the temporal neural differential equation module. Specifically, this includes: using the Euler trapezoidal integral method to solve the forward propagation equation of the graph convolutional neural network based on nonlinear state feedback, obtaining the evolution state after k propagations; and then... k The evolutionary state after the first propagation serves as the initial value for evolution at the temporal level.

[0058] More specifically, addressing the feature oversmoothing phenomenon in deep GCNs, this application combines a state feedback mechanism with graph ordinary differential equations (ODEs), enhancing the model's representational power while suppressing feature oversmoothing. Spatial neural differential equations (NDEs) model the information propagation mechanism on the graph as an NDE and simulate the information propagation process on the graph using the forward Euler integral method. Compared to traditional graph ODEs, spatial NDEs can utilize feedback functions... The outputs of each layer of the GCN are adaptively adjusted to overcome the over-smoothing of features during depth map message passing. The forward propagation formula for the graph ODE is as follows: The proposed forward propagation formula for spatial NDE is: .

[0059] Compared to traditional graphical ODEs, spatial neural differential equation modules can utilize feedback functions. The outputs of each layer of the GCN are adaptively adjusted to overcome the over-smoothing of features during depth map message passing. The forward propagation formula for the graph ODE is as follows: The proposed forward propagation formula for spatial NDE is: .

[0060] Specifically, we first define the information propagation on the graph (graph structure), as shown in formula (3).

[0061] (3)

[0062] in, A system matrix representing the laws governing information propagation on a graph. It is the input control matrix. It is system input. For the output matrix, For the first k The state after secondary information aggregation. Therefore, it has a control rate. The status feedback on the graph is as follows:

[0063] (4)

[0064] in, For the state feedback matrix, and These are the system's reference input and the control matrix for the reference input, respectively. Of course, external factors such as weather, i.e., the input... It exists, and the parameter matrix can be learned. Otherwise, Assign a value of 0 and ignore. The impact. If not specified, set it in the settings. =0. For classic GCN, use This is used to adjust the output of each GCN layer to prevent each node's features from becoming too similar due to the aggregation of too much neighbor information.

[0065] because and Essentially, linear matrices are used, but the spatial interactions of multivariate time series are not merely simple linear relationships; they also possess certain nonlinear characteristics. Therefore, using linear matrices to model nonlinear spatial features will inevitably lead to modeling bias. To address this issue, neural networks are chosen. Replace the original two linear matrices and This is used to model the spatial characteristics in multivariate time series. Therefore, the forward propagation equation of GCN based on nonlinear state feedback is:

[0066] (5)

[0067] in, Indicates the first k The state of a layered graph convolutional neural network. Represents the adjacency matrix. The first expression of the spatial neural differential equation k Step evolution state, Represents the feedback function. Both are non-linear activation functions with input and output dimensions of 1 / 2. d linear layer, and These are the reference input control matrix and the reference output control matrix of the target nonlinear dynamic system, respectively. This is the input control matrix.

[0068] After modeling the traditional GCN into a spatial NDE, the Euler trapezoidal integral method is used to solve formula (5) and obtain the state after k propagations.

[0069] Finally, k Multiplying the hidden state of a layer GCN by the weight of the corresponding layer yields... k The final result of layer graph convolution: (6)

[0070] in, This serves as the initial spatiotemporal evolution state of the entire dynamic system and is fed into the time NDE. Indicates the first k The learnable output matrix of the layer GCN. Therefore, the propagation number k It will be replaced by the time step t in the time NDE, that is .

[0071] In an exemplary embodiment, based on the initial value of the time-level evolution, the multidimensional state quantity corresponding to the multidimensional input data at the next moment is obtained through state transition. Specifically, this includes: using the Euler trapezoidal integral method to solve the mechanism evolution equation of the time-level evolution to obtain the multidimensional state quantity corresponding to the multidimensional input data at the next moment.

[0072] After obtaining the initial representation of the dynamic system, this application takes a mechanistic modeling approach rather than fitting labels and training, deducing the future state of the dynamic system in a high-dimensional space. SF-GDE benefits from Koopman operator theory, using neural networks instead of Koopman linear operators to model the evolution mechanism of the dynamic system. Furthermore, based on nonlinear state transition theory, a time-based NDE... As the initial driving force, the dynamic evolution process of the dynamic system is simulated by the Euler trapezoidal integral method, and the high-dimensional state with the required prediction time step is obtained through state transition.

[0073] Specifically, the mechanistic evolution equation of a dynamic system is defined as: (7)

[0074] in, This represents the evolution mechanism of a dynamic system. To accurately learn the nonlinear dynamics of a dynamic system, The sandwich network structure consists of a second linear layer, a Long Short-Term Memory (LSTM) network, and a third linear layer connected sequentially. The second linear layer maps the input data to a 2D high-dimensional space, and then the LSTM maps the data to a 4D high-dimensional space. The purpose is to linearize the nonlinear features, enabling the time-based NDE to learn the system mechanism more accurately. Finally, the second linear layer maps the 4D spatial data back to the original D-dimensional space. After establishing the system equations, considering computational complexity and solution accuracy, the Euler trapezoidal integral method is chosen to solve the system equations. The solution result is shown in formula (8):

[0075] (8)

[0076] Where T represents transpose. Indicates the future The high-dimensional state of a time-step dynamic system For the integration interval, To predict the step size. Observing formula (8), we can see that in order to solve for the future... To determine the higher-dimensional state of the system, we first need to know the system's position. The state at any given moment, i.e. However, in time series prediction tasks, the high-dimensional state at future times is unknown. Therefore, a predictor-corrector model is introduced to approximate the system's state in the future. The state at time t is given by the following formula:

[0077] (9)

[0078] Substituting formula (9) into formula (8) yields:

[0079] (10)

[0080] because The essence of is a real number that can be learned by a neural network, so formula (10) is simplified to:

[0081] (11)

[0082] in, for Multidimensional state variables at time. for Multidimensional state variables at time. for Multidimensional state variables at time. t For time, To predict the time step.

[0083] The decoder receives the future high-dimensional state of the dynamic system inferred through time-based NDE. Since SF-GDE is a mechanism-driven model, it does not require the design of a complex decoder network. For this reason, this application chooses to use the connection layer and linear layer of LSTM as the decoder. LSTM decodes the feature dimension D to D / 2, and the linear layer decodes the feature dimension D / 2 to 1, obtaining the final medium- and long-term prediction values. .

[0084] Considering that the prediction mechanism of SF-GDE mainly simulates the spatiotemporal dynamic mechanism evolution of dynamic systems through spatiotemporal NDE, it is necessary to ensure the stability of the model during the evolution process. Therefore, this application applies the Lyapunov second method stability criterion to the entire model. Thus, as shown in Equation 12, the loss function of SF-GDE consists of two parts: the prediction task and the stability criterion.

[0085] The loss function for training the network model is expressed as: (12)

[0086] in, L This is the total loss value. For mean square error loss, Losses due to the stability criteria.

[0087] For prediction tasks, the formula is as follows:

[0088] (13)

[0089] in, For predicted values, For tags, This represents the number of samples.

[0090] For the stability criterion, the model must satisfy four conditions. First, the model must have zero initial conditions. That is, the initial state of the system at time step zero is zero, where This represents the state at time step zero. This represents a system function, here. This represents the entire model, including the spatial NDE and temporal NDE mentioned above. Therefore, it is added to the loss function. This ensures that the SF-GDE satisfies the zero initial conditions. Secondly, after satisfying the zero initial conditions, a positive definite energy function V( x ), such that V(0) = 0. To satisfy the second condition, the energy function is defined as Third, the Lyapunov stability criterion requires that the derivative of the energy function be negative definite, i.e. Therefore, Add to the loss function to make Always less than 0, where These are hyperparameters that are artificially selected and are greater than 0. The larger, The more strictly negative the definiteness, the more stringent the condition. Fourth, as the magnitude of X approaches infinity, the energy function... It also tends towards infinity, that is Sometimes, In summary, the loss function for the stability criterion is in the following form:

[0091] (14)

[0092] in, Represents system functions, The state at time step zero. For ReLU activation functions, Energy function The derivative of , Indicates the first n The characteristics of each node N is the number of nodes in the adaptive graph module. This is a hyperparameter.

[0093] The introduction of the Lyapunov stability criterion ensures the model remains stable during mechanistic evolution. This application also compares the proposed prediction model with state-of-the-art techniques in atmospheric and traffic flow fields, demonstrating its superiority.

[0094] The structure of the SF-GDE model, i.e., the multivariate time series prediction model, is as follows: Figure 3 As shown, the adaptive graph module compresses the original high-dimensional input data X into a low-dimensional representation. By calculating the cosine similarity between the low-dimensional representations, it adaptively mines the interaction relationships between nodes, providing the encoder module with an adjacency matrix A that can represent the interaction relationships between nodes.

[0095] The encoder's role is to map the original high-dimensional input data X to a high-dimensional space, aiming to linearize the nonlinear spatiotemporal features in the original data, making the model easier to learn. The encoder consists of a simple linear layer and a traditional GCN layer, designed to provide initial values ​​for the spatial evolution of the spatial NDE module. .

[0096] The role of the spatial NDE module is to model the information propagation mechanism on the graph as an NDE, and to simulate the information propagation process on the graph using the forward Euler integral method. This aims to fully integrate spatial features while suppressing the oversmoothing of features in deep GCNs, and to provide initial values ​​for the temporal evolution of the temporal NDE module. .

[0097] The role of the time-based NDE module is to model the operating mechanism of complex nonlinear systems, learn and establish the system's mechanistic equations. Subsequently, the dynamic evolution process of the dynamic system is simulated using the Euler trapezoidal integral method, and high-dimensional states for the desired prediction time step are obtained through state transitions. .

[0098] The decoder's role is to decode the high-dimensional state obtained by solving the time-based NDE, with the aim of obtaining the final predicted value Y.

[0099] The target nonlinear dynamic system is a traffic system, which includes multiple monitoring nodes. The multivariate time series prediction model is used to predict the traffic flow of each monitoring node at the next moment based on the traffic flow of each monitoring node at the previous moment.

[0100] In an exemplary embodiment, to demonstrate the superiority of SF-GDE in medium- to long-term time series forecasting tasks, comparative experiments were conducted on the Seoul PM2.5 dataset, the PEMS04 traffic flow dataset, and the Beijing temperature dataset. Specific information about the historical datasets is shown in Table 1. All experiments were performed on a 12th-generation Intel(R) Core(TM) i7-12800HX CPU and an NVIDIA GeForce RTX 3080Ti laptop GPU. All parameters of the SF-GDE model were set as follows: number of nodes N determined by the dataset, embedding dimension d=10, input history step size h=5, feature dimension d=16, propagation times k=10, and prediction step size τ=10. SF-GDE first learns the graph structure by calculating 10-dimensional node embedding vectors. Then, it performs 10 graph convolution operations using spatial NDE to map the 5-step historical data into a 16-dimensional space. Finally, it obtains the predicted values ​​for the next 10 steps through temporal NDE and a decoder.

[0101] During training, the PyToch framework was used, with a batch size of 64. The Adom optimization algorithm with an initial learning rate of 0.001 was chosen to optimize the learnable parameters in the SF-GDE. Hyperparameters in the loss function... The training iterations were set to 2000, with a value of 0.1. All datasets were divided into training and test sets in a 4:1 ratio. In the comparative experiments, Root Mean Square Error (RMSE), Mean Absolute Error (MAE), and Symmetric Mean Absolute Percentage Error (SMAPE) losses were used as evaluation metrics for the models. Furthermore, to demonstrate the superiority of the SF-GDE model, this application selected the iTransformer model, published in the top-tier computer science conference ICLR in 2024, as a comparison algorithm. The iTransformer model utilizes a feedforward network to capture temporal features and an attention mechanism to capture the interactions between sequences, achieving spatiotemporal time series prediction.

[0102] Table 1 Dataset Information

[0103]

[0104] As shown in Table 2, the proposed algorithm SF-GDE achieves optimal performance across most metrics. Specifically, iTransformer modifies the traditional Transformer framework, treating independent time series as tokens. It captures correlations between variables through self-attention and utilizes layer normalization and feedforward neural networks to model global time dependencies, thus modeling both the interactions between variables and the overall time dependencies. However, like Transformer-like models, the lack of modeling for feature locations inevitably leads to modeling errors.

[0105] Table 2 Seoul PM 2.5 PEMS04 and Beijing temperature 10-step forecast (best indicators are shown in bold, second-best indicators are shown in underline)

[0106]

[0107] The SF-GDE (Linear) model's network consists of purely linear layers, while SF-GDE (Non-linear) adds a non-linear activation function to the linear layers. Observing the prediction performance of the two models on three datasets reveals that the SF-GDE (Linear) model achieves a slight advantage in short-term predictions, but its performance in medium- and long-term predictions is far inferior to the SF-GDE (Non-Linear) model. This is because current multivariate time series data exhibits strong non-linear characteristics, and state feedback is a control method for linear systems, thus leading to certain modeling errors. However, the prediction performance of the SF-GDE (Linear) model is still significantly better than other comparative algorithms. This demonstrates that the introduction of state feedback provides a better theoretical basis for the model and can effectively improve its prediction performance. Furthermore, considering that engineering applications often require predictions for multiple future time steps, this application combines state feedback with a non-linear activation function and achieves better prediction performance.

[0108] Unlike most time series forecasting models, the mechanism-driven SF-GDE model models the spatiotemporal evolution mechanism of dynamic systems through neural networks and simulates the mechanistic evolution at both the spatial and temporal levels using the Euler forward integral method and the Euler trapezoidal method, respectively. Furthermore, SF-GDE obtains the future state of a system by simulating its mechanistic evolution rather than by fitting training data and labels, thus achieving better predictive performance on three datasets.

[0109] In summary, this application proposes a mechanism-driven spatiotemporal time series modeling method to reveal the dynamic evolution patterns of spatiotemporal features and to predict time series. By compressing high-dimensional sequences into low-dimensional embeddings and simulating the propagation of information and feature aggregation processes on the graph using a state feedback-based GDE, the SF-GDE model overcomes the limitations of data dimensionality and alleviates the oversmoothing phenomenon of deep GCNs. From the perspective of nonlinear system modeling, this is a novel mechanism modeling strategy. Utilizing nonlinear state transition equations and the Lyapunov stability criterion, SF-GDE can discover key internal states in spatiotemporal time series and remain stable during the simulation of spatiotemporal mechanism evolution, providing a method with a certain degree of interpretability for fully exploring the dynamic characteristics of nonlinear complex systems. Experimental results show that this framework is an effective method for revealing nonlinear dynamic systems and predicting multivariate time series.

[0110] The technical advantages of this application are as follows: 1. A joint framework for nonlinear time series modeling and automatic mining of latent spatial topology is proposed. From the perspective of complex nonlinear system modeling, neural differential equations based on state feedback are combined with neural differential equations based on spatial state description to model the spatiotemporal characteristics of multivariate time series. 2. By introducing Lyapunov stability constraints, the asymptotic stability of the model over a large range is guaranteed, enhancing the interpretability of the model and improving the prediction performance. 3. Comparative experiments were conducted to verify the effectiveness of the proposed model.

[0111] Based on the same inventive concept, this application also provides a multivariate time series forecasting apparatus for implementing the multivariate time series forecasting method described above. The solution provided by this apparatus is similar to the implementation scheme described in the above method; therefore, the specific limitations in one or more embodiments of the multivariate time series forecasting apparatus provided below can be found in the limitations of the multivariate time series forecasting method described above, and will not be repeated here.

[0112] In one exemplary embodiment, this application provides a multivariate time series prediction apparatus comprising:

[0113] A network model construction module is used to construct a network model based on a graph convolutional neural network according to the topology of the target nonlinear dynamic system. The network model includes an adaptive graph module, an encoder, a spatial neural differential equation module, a temporal neural differential equation module, and a decoder connected in sequence. The adaptive graph module is determined based on the topology of the target nonlinear dynamic system.

[0114] The network model training module is used to train the network model using historical datasets of the target nonlinear dynamic system to obtain a multivariate time series prediction model; each sample in the historical dataset is a multivariate time series. The adaptive graph module is used to compress multidimensional input data and provide an adjacency matrix for the encoder. The encoder is used to extract nonlinear spatiotemporal features from the multidimensional input data based on the adjacency matrix, providing initial values ​​for spatial evolution for the spatial neural differential equation module. The spatial neural differential equation module is used to provide initial values ​​for temporal evolution for the temporal neural differential equation module by combining a state feedback mechanism and graph ordinary differential equations, fusing spatial features while suppressing feature oversmoothing, based on the initial values ​​for spatial evolution. The temporal neural differential equation module is used to obtain the multidimensional state variables corresponding to the next time step of the multidimensional input data through state transition based on the initial values ​​for temporal evolution. The decoder is used to decode the multidimensional state variables to obtain predicted values.

[0115] The prediction module is used to predict the multivariate time data of the next time step based on the multivariate time data of the previous time step using the multivariate time series prediction model.

[0116] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0117] This document uses specific examples to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the methods and core ideas of this application. Furthermore, those skilled in the art will recognize that, based on the ideas of this application, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A method for multivariate time series forecasting, the method comprising: The multi-element time series prediction method comprises: According to the topological structure of the target nonlinear dynamic system, a network model based on a graph convolutional neural network is constructed; the network model comprises an adaptive graph module, an encoder, a spatial neural differential equation module, a time neural differential equation module and a decoder connected in sequence; the adaptive graph module is determined based on the topological structure of the target nonlinear dynamic system; The network model is trained using a historical data set of the target nonlinear dynamic system to obtain a multi-element time series prediction model; each sample in the historical data set is a multi-element time series; the adaptive graph module is used to compress multi-dimensional input data and provide an adjacency matrix for the encoder; the encoder is used to extract nonlinear space-time features in the multi-dimensional input data based on the adjacency matrix and provide initial values of spatial level evolution for the spatial neural differential equation module; the spatial neural differential equation module is used to combine a state feedback mechanism with a graph ordinary differential equation based on the initial values of spatial level evolution, fuse spatial features while suppressing feature oversmoothing, and provide initial values of time level evolution for the time neural differential equation module; the time neural differential equation module is used to obtain a multi-dimensional state quantity corresponding to the next time of the multi-dimensional input data through state transition based on the initial values of time level evolution; and the decoder is used to decode the multi-dimensional state quantity to obtain a prediction value; The multi-element time series prediction model is used to predict multi-element time data at the next time according to multi-element time data at the previous time.

2. The multi-variate time series prediction method of claim 1, wherein, Each node in the adaptive graph module is assigned an adaptive embedding vector, which is used to represent low-dimensional data after compression of multi-dimensional input data.

3. The multi-variate time series prediction method of claim 2, wherein, The adjacency matrix is represented as: wherein A represents an adjacency matrix, a ij is the connection weight of the edge between the ith node and the jth node, T represents a transpose, e i represents the adaptive embedding vector of the ith node, and N is the number of nodes.

4. The multi-variate time series prediction method of claim 1, wherein, The encoder comprises a first linear layer and a graph convolutional neural network.

5. The multi-variate time series prediction method of claim 4, wherein, The initial values of spatial level evolution are represented as: wherein, denotes the k-th step evolution state of the spatial neural differential equation, R N×D is an N x D dimensional data, X ∈ R N ×h is a multi-dimensional input data, R N×h is an N x h dimensional data, N is the number of nodes in the adaptive graph module, h is the step size, D is the feature dimension, W k ∈ R h×D is a learnable parameter of the k-th layer graph convolutional neural network.

6. The multi-variate time series prediction method of claim 1, wherein, Based on the initial values of spatial level evolution, a state feedback mechanism is combined with a graph ordinary differential equation to fuse spatial features while suppressing feature oversmoothing, and the time neural differential equation module is provided with initial values of time level evolution, specifically comprising: An Euler trapezoidal integration method is used to solve the forward propagation equation of the graph convolutional neural network based on nonlinear state feedback to obtain an evolution state after k times of propagation; The evolution state after the kth time of propagation is taken as the initial value of time level evolution; The forward propagation equation of the graph convolutional neural network based on nonlinear state feedback is represented as: Wherein, H(k) represents the state of the kth layer graph convolutional neural network, A represents the adjacency matrix, H k G represents the kth step evolution state of the spatial neural differential equation, f feedback represents the feedback function, v and L are respectively the reference input control matrix and the reference output control matrix of the target nonlinear dynamic system, and B is the input control matrix.

7. The multi-variate time series prediction method of claim 1, wherein, Based on the initial values of time level evolution, a multi-dimensional state quantity corresponding to the next time of the multi-dimensional input data is obtained through state transition, specifically comprising: An Euler trapezoidal integration method is used to solve the mechanism evolution equation of the time level evolution to obtain a multi-dimensional state quantity corresponding to the next time of the multi-dimensional input data; The multi-dimensional state quantity at the next time is represented as: wherein T denotes transpose, a multi-dimensional state quantity at a time t+τ-1, a multi-dimensional state quantity at a time t+τ-1, a multi-dimensional state quantity at a time t+τ-2, t is time, τ is a prediction time step, f() is a sandwich network structure, and the sandwich network structure comprises a second linear layer, a long short-term memory network and a third linear layer connected in sequence.

8. The multi-variate time series prediction method of claim 1, wherein, The loss function when training the network model is represented as: L = L pred + L stable ; wherein L is a total loss value, L pred is a mean square error loss, L stable is a loss for the stability criterion part; wherein f represents a system function, x(0) is a state at a zero time step, and Relu() is a Relu activation function, is a derivative of the energy function V(x), x n represents a feature of the nth node, 1≤n≤N, and N is a number of nodes in the adaptive graph module.

9. The multi-variate time series prediction method of claim 1, wherein, The target nonlinear dynamic system is a traffic system, and the traffic system comprises a plurality of monitoring nodes; the multi-element time series prediction model is used to predict traffic flow of each monitoring node at the next time according to traffic flow of each monitoring node at the previous time.

10. A multi-variate time series prediction device, characterized by comprising: The multi-element time series prediction device comprises: The network model construction module is configured to construct a network model based on a graph convolutional neural network according to a topological structure of the target nonlinear dynamic system; the network model comprises an adaptive graph module, an encoder, a spatial neural differential equation module, a temporal neural differential equation module and a decoder connected in sequence; the adaptive graph module is configured to determine based on the topological structure of the target nonlinear dynamic system; The network model training module is configured to train the network model by using a historical data set of the target nonlinear dynamic system to obtain a multivariate time series prediction model; each sample in the historical data set is a multivariate time series; the adaptive graph module is configured to compress multivariate input data and provide an adjacency matrix for the encoder; the encoder is configured to extract nonlinear space-time features in the multivariate input data based on the adjacency matrix and provide initial values of space-level evolution for the spatial neural differential equation module; the spatial neural differential equation module is configured to combine a state feedback mechanism with a graph ordinary differential equation based on the initial values of space-level evolution, fuse spatial features and suppress feature oversmoothing, and provide initial values of time-level evolution for the temporal neural differential equation module; the temporal neural differential equation module is configured to obtain a next time multivariate state variable corresponding to the multivariate input data by state transition based on the initial values of time-level evolution; and the decoder is configured to decode the multivariate state variable to obtain a prediction value; The prediction module is configured to predict multivariate time data at a next time according to multivariate time data at a previous time by using the multivariate time series prediction model.

Images