Spatial-temporal data prediction method, device and equipment

Abstract

The invention provides a spatio-temporal data prediction method, device and equipment, which can be applied to the technical field of artificial intelligence. The method comprises the steps that a to-be-measured space-time diagram is input into a space-time diagram convolutional network, a target vector used for representing the to-be-measured space-time diagram, a prediction label vector of the target vector and prediction space-time data corresponding to the prediction label vector are obtained, the prediction label vector is obtained by predicting the target vector, and the prediction label vector is obtained by predicting the target vector; the prediction spatio-temporal data is obtained after reverse quantization is carried out on the prediction label vector, and the spatio-temporal diagram to be measured comprises a plurality of nodes and edges used for representing the incidence relation among the nodes; inputting the target vector and the prediction label vector into a space-time residual information propagation module to obtain target residual data representing prediction deviation of prediction space-time data; and correcting the predicted spatio-temporal data based on the target residual data to obtain target prediction data for the to-be-measured spatio-temporal diagram.

Description

Technical Field

[0001] The present disclosure relates to the field of artificial intelligence technology, and specifically to a spatiotemporal data prediction method, device and equipment. Background Art

[0002] In real life, massive amounts of spatiotemporal data are generated every moment. Predicting future spatiotemporal data can enable advance knowledge of future situations, so as to facilitate prevention or control based on the prediction results of actual scenarios. This plays an important role in the fields of intelligent transportation, weather forecasting, etc.

[0003] In the related art, the spatiotemporal data of the historical period is usually input into the spatiotemporal prediction model to obtain the spatiotemporal data of the future period. However, many spatiotemporal prediction models are based on an assumption in operation that there are consistent spatiotemporal features between the input data and the future prediction data. These spatiotemporal prediction models rely on the spatiotemporal features extracted from the input data to generate future prediction data. Since achieving complete consistency between the input data and the future prediction data is an idealized goal, in practice there are often spatiotemporal deviations, resulting in the problem of low accuracy of future prediction data. Summary of the invention

[0004] In view of the above problems, the present disclosure provides a spatiotemporal data prediction method, apparatus, device, medium and program product.

[0005] According to one aspect of the present disclosure, a method for predicting spatiotemporal data is provided, comprising: inputting a spatiotemporal graph to be tested into a spatiotemporal graph convolutional network, obtaining a target vector for representing the spatiotemporal graph to be tested, a predicted label vector for the target vector, and predicted spatiotemporal data corresponding to the predicted label vector, wherein the predicted label vector is obtained by predicting the target vector, and the predicted spatiotemporal data is obtained by inverse quantization of the predicted label vector, and the spatiotemporal graph to be tested includes a plurality of nodes and edges for characterizing the association relationship between the plurality of nodes; inputting the target vector and the predicted label vector into the In the spatiotemporal residual information propagation module, target residual data characterizing the prediction deviation of the predicted spatiotemporal data is obtained; the predicted spatiotemporal data is corrected based on the target residual data to obtain target prediction data for the spatiotemporal graph to be tested; wherein, the spatiotemporal residual information propagation module is determined based on the Gauss-Markov hypothesis model, and the target residual data includes target residual sub-data of each of the multiple nodes, and the spatiotemporal residual information propagation module is used to determine the target residual sub-data of each of the multiple nodes based on the deviation vector between the target vector and the predicted label vector and the correlation relationship between the multiple nodes.

[0006] Another aspect of the present disclosure provides a spatiotemporal data prediction device, including: an initial prediction module, used to input a spatiotemporal graph to be tested into a spatiotemporal graph convolutional network, and obtain a target vector for representing the spatiotemporal graph to be tested, a predicted label vector for the target vector, and predicted spatiotemporal data corresponding to the predicted label vector, wherein the predicted label vector is obtained by predicting the target vector, and the predicted spatiotemporal data is obtained by inverse quantization of the predicted label vector, and the spatiotemporal graph to be tested includes a plurality of nodes and edges for characterizing the association relationship between the plurality of nodes; a residual prediction module, used to input the target vector and the predicted label vector into a convolutional network; To the spatiotemporal residual information propagation module, target residual data characterizing the prediction deviation of the predicted spatiotemporal data are obtained, wherein the spatiotemporal residual information propagation module is determined based on the Gauss-Markov hypothesis model, and the target residual data include target residual sub-data of each of the multiple nodes, and the spatiotemporal residual information propagation module is used to determine the target residual sub-data of each of the multiple nodes based on the deviation vector between the target vector and the predicted label vector and the correlation relationship between the multiple nodes; a correction module is used to correct the predicted spatiotemporal data based on the target residual data to obtain target prediction data for the spatiotemporal graph to be tested.

[0007] Another aspect of the present disclosure provides an electronic device, comprising: one or more processors; and a memory for storing one or more computer programs, wherein the one or more processors execute the one or more computer programs to implement the steps of the above method.

[0008] The spatiotemporal data prediction method disclosed in the present invention realizes vectorization of the spatiotemporal graph to be tested to obtain a target vector, backward prediction to obtain a predicted label vector, and inverse quantization to obtain predicted spatiotemporal data by inputting the spatiotemporal graph to be tested into a spatiotemporal graph convolutional network, and inputs the target vector and the predicted label vector into a spatiotemporal residual information propagation module to obtain target residual data that characterizes the prediction deviation of the predicted spatiotemporal data, and corrects the predicted label data using the target residual data to obtain target predicted data. By inputting the target vector and the predicted label data into the spatiotemporal residual information propagation module, the spatiotemporal residual information propagation module pays attention to the spatiotemporal deviation between the input data and the future predicted data. Specifically, since the spatiotemporal graph to be tested includes multiple nodes and edges for characterizing the association relationship between the multiple nodes, the target residual data includes target residual sub-data of each of the multiple nodes. In the spatiotemporal residual information propagation module, the target residual sub-data of each of the multiple nodes are determined by the association relationship between the multiple nodes and the deviation vector between the target vector and the predicted label vector, so that the residual sub-data of the current node is determined through other nodes, thereby improving the accuracy of the residual sub-data of each node, and correcting the predicted label data through the target residual data, at least partially solves the problem of prediction accuracy existing in the related technology, and achieves the technical effect of more accurate prediction of future data. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] The above contents and other objects, features and advantages of the present disclosure will become more apparent through the following description of the embodiments of the present disclosure with reference to the accompanying drawings, in which:

[0010] Figure 1 A flowchart of a spatiotemporal data prediction method according to an embodiment of the present disclosure is schematically shown;

[0011] Figure 2 The model architecture diagram of the spatiotemporal data prediction method according to the embodiment of the present disclosure is schematically shown;

[0012] FIG3 (a) schematically shows a first implementation result diagram of the spatiotemporal data prediction method according to an embodiment of the present disclosure;

[0013] FIG3( b ) schematically shows a second implementation result diagram of the spatiotemporal data prediction method according to an embodiment of the present disclosure;

[0014] FIG3 (c) schematically shows a third implementation result diagram of the spatiotemporal data prediction method according to an embodiment of the present disclosure;

[0015] FIG4( a ) schematically shows a first visual diagram of the convergence of a spatiotemporal graph convolutional network and a spatiotemporal residual information propagation module when training the spatiotemporal graph convolutional network and the spatiotemporal residual information propagation module using a first data set according to an embodiment of the present disclosure;

[0016] FIG4( b ) schematically shows a second visual diagram of the convergence of the spatiotemporal graph convolutional network and the spatiotemporal residual information propagation module when the first data set is used to train the spatiotemporal graph convolutional network according to an embodiment of the present disclosure;

[0017] FIG4( c ) schematically shows a third visual diagram of the convergence of the spatiotemporal graph convolutional network and the spatiotemporal residual information propagation module when the first data set is used to train the spatiotemporal graph convolutional network according to an embodiment of the present disclosure;

[0018] FIG4( d ) schematically shows a fourth visual diagram of the convergence of the spatiotemporal graph convolutional network and the spatiotemporal residual information propagation module when the first data set is used to train the spatiotemporal graph convolutional network according to an embodiment of the present disclosure;

[0019] Figure 5 A visualization diagram schematically shows the convergence of the spatiotemporal graph convolutional network and the spatiotemporal residual information propagation module when the second data set is used to train the spatiotemporal graph convolutional network according to an embodiment of the present disclosure;

[0020] Figure 6 A schematic diagram of a structure of a spatiotemporal data prediction device according to an embodiment of the present disclosure is shown; and

[0021] Figure 7 A block diagram of an electronic device suitable for implementing a spatiotemporal data prediction method according to an embodiment of the present disclosure is schematically shown. DETAILED DESCRIPTION

[0022] Hereinafter, embodiments of the present disclosure will be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of the present disclosure. In the following detailed description, for ease of explanation, many specific details are set forth to provide a comprehensive understanding of the embodiments of the present disclosure. However, it is apparent that one or more embodiments may also be implemented without these specific details. In addition, in the following description, descriptions of known structures and technologies are omitted to avoid unnecessary confusion of the concepts of the present disclosure.

[0023] The terms used herein are only for describing specific embodiments and are not intended to limit the present disclosure. The terms "comprise", "include", etc. used herein indicate the existence of the features, steps, operations and / or components, but do not exclude the existence or addition of one or more other features, steps, operations or components.

[0024] All terms (including technical and scientific terms) used herein have the meanings commonly understood by those skilled in the art unless otherwise defined. It should be noted that the terms used herein should be interpreted as having a meaning consistent with the context of this specification and should not be interpreted in an idealized or overly rigid manner.

[0025] When using expressions such as "at least one of A, B, and C, etc.", they should generally be interpreted according to the meaning of the expression commonly understood by those skilled in the art (for example, "a system having at least one of A, B, and C" should include but is not limited to a system having A alone, B alone, C alone, A and B, A and C, B and C, and / or A, B, C, etc.).

[0026] It should be noted that the prediction method and device for determining spatiotemporal data disclosed in the present invention can be used in the field of computer technology, and can also be used in any field other than the field of computer technology, such as the field of artificial intelligence technology. The present invention does not limit the application field of the prediction method and device for spatiotemporal data.

[0027] In the technical solution of the present disclosure, the data involved (including but not limited to the spatiotemporal data to be tested for analysis, stored data, displayed data, etc.) are all information and data authorized by the user or fully authorized by all parties, and the collection, storage, use, processing, transmission, provision, disclosure and application of the relevant data comply with relevant laws, regulations and standards, take necessary confidentiality measures, do not violate public order and good morals, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0028] Spatio-Temporal Graph Convolutional Network (STGNNs) has attracted great attention from academia and industry as a popular method for spatio-temporal prediction tasks. Currently, the methodological focus of spatio-temporal graph convolutional networks is to create more expressive algorithmic architectures compared to basic variants such as graph convolutional networks (GCN) or self-attention mechanisms (Transformer). However, as the complexity of the model increases, the potential for performance improvement becomes minimal, and how to properly utilize label information has not yet been explored in this field.

[0029] Label propagation, as a semi-supervised learning method, has been widely used in the field of graph learning. For example, GFHF uses a Gaussian random field model to formulate a label propagation framework. PPNP (Probabilistic Personalized PageRank with Neural Predictions) and APPNP (Adaptive Personalized PageRank with Neural Predictions) add a graph neural network (GNN) module to enhance the label prediction ability. However, they only use label information or feature information in the training and inference phases. GCN-LPA (Graph Convolutional Network with Label Propagation and Attention) uses GNN (Graph Neural Networks) and Page-Rank to use both feature and label information in training. UniMP uses a self-attention mechanism structure to fuse features and labels simultaneously in the training and inference phases. UGM-LGC theorizes and frames the above phenomena in a unified way. ILP (Incremental Learning with Pseudo-Labels) overcomes the imbalance of input labels by incrementally adding credible pseudo-labels to balance the distribution of labels. Since the labels of all test data are unknown, we cannot directly apply these semi-supervised algorithms to the field of spatiotemporal graph prediction.

[0030] In recent years, the spatiotemporal graph convolutional network is the most representative spatiotemporal prediction model in spatiotemporal prediction tasks. It usually includes a spatial module and a temporal module, the former captures spatial dependencies and the latter captures temporal dependencies. For example, D2STGNN (Decoupled Dynamic Spatial-Temporal Graph Neural Network for Traffic Forecasting) uses diffusion graph convolutional networks and recurrent neural networks (RNNs) to capture temporal patterns. SSTBAN follows a multi-task framework and incorporates a self-supervised learner to generate robust potential representations for historical traffic data. And such as Spatio-Temporal Interaction Detection (STID) and Dynamic Directed Graph Convolutional Recurrent Network (DDGCRN) use prediction information to enhance spatiotemporal modeling capabilities and claim to eliminate abnormal signals.

[0031] However, the inventors of the present disclosure have found in the course of their research that many models in the related art are based on an assumption in their operation that there are consistent spatiotemporal features (i.e., correlation) between the input data and the future prediction data. These models rely on the spatiotemporal features extracted from the input data to directly generate the future prediction data. However, this assumption has no theoretical support because it is an idealized goal to achieve complete consistency between the input data and the future prediction data, especially considering the existence of spatiotemporal deviations.

[0032] Therefore, an embodiment of the present disclosure provides a spatiotemporal data prediction method, which inputs a spatiotemporal graph to be tested into a spatiotemporal graph convolutional network to obtain a target vector for representing the spatiotemporal graph to be tested, a predicted label vector of the target vector, and predicted spatiotemporal data corresponding to the predicted label vector; inputs the target vector and the predicted label vector into a spatiotemporal residual information propagation module to obtain target residual data that characterizes the prediction deviation of the predicted spatiotemporal data; and corrects the predicted spatiotemporal data based on the target residual data to obtain target prediction data for the spatiotemporal graph to be tested.

[0033] Figure 1 The flowchart of the spatiotemporal data prediction method according to an embodiment of the present disclosure is schematically shown.

[0034] like Figure 1 As shown, the method includes operations S110 to S130.

[0035] In operation S110, the spatiotemporal graph to be tested is input into the spatiotemporal graph convolutional network to obtain a target vector representing the spatiotemporal graph to be tested, a predicted label vector of the target vector, and predicted spatiotemporal data corresponding to the predicted label vector, wherein the predicted label vector is obtained by predicting the target vector, the predicted spatiotemporal data is obtained by inverse quantization of the predicted label vector, and the spatiotemporal graph to be tested includes multiple nodes and edges for characterizing the association relationship between the multiple nodes.

[0036] In operation S120, the target vector and the predicted label vector are input into the spatiotemporal residual information propagation module to obtain target residual data representing the prediction deviation of the predicted spatiotemporal data, wherein the spatiotemporal residual information propagation module is determined based on the Gauss-Markov hypothesis model, and the target residual data includes target residual sub-data of each of the multiple nodes. The spatiotemporal residual information propagation module is used to determine the target residual sub-data of each of the multiple nodes based on the deviation vector between the target vector and the predicted label vector and the correlation relationship between the multiple nodes.

[0037] In operation S130, the predicted spatiotemporal data is corrected based on the target residual data to obtain target predicted data for the spatiotemporal graph to be tested.

[0038] According to the embodiments of the present disclosure, there is no limitation on the spatio-temporal graph convolutional network, and it can be any network that can predict the spatio-temporal graph, such as STGNNs (Spatio-Temporal Graph Neural Networks) and the like.

[0039] According to the embodiments of the present disclosure, the predicted spatiotemporal data may be spatiotemporal data, i.e., data including two dimensions of time and space, for example: including observation objects, time series data of observation objects, etc. Specifically, in a traffic scenario, the observation object may be a road detection point, and the time series data may be the traffic volume at the road detection point over a period of time.

[0040] According to an embodiment of the present disclosure, the time-space diagram to be measured may be data of a historical time period, and the predicted time data may be data of a future time period of the historical time period.

[0041] According to an embodiment of the present disclosure, the spatiotemporal graph convolutional network may include an encoder, a spatiotemporal learning layer, and a decoder, and they are serially connected in the order of the encoder, the spatiotemporal learning layer, and the decoder. The encoder, the spatiotemporal learning layer, and the decoder are not limited and can be determined according to the neural network model used.

[0042] According to an embodiment of the present disclosure, the spatiotemporal residual information propagation module may be obtained through a Gauss-Markov hypothesis model. Specifically, the construction process of the Gauss-Markov hypothesis model includes the following steps.

[0043] According to an embodiment of the present disclosure, the time-space diagram to be tested can be Represents, where V represents a node set with N nodes, represents the edge set, is the space-time graph to be tested The weighted adjacency matrix of is a set of real numbers. At time step t, the spatiotemporal graph data of the f observable features of all N nodes is represented as When using the spatiotemporal graph convolutional network for prediction, input a spatiotemporal graph to be tested To represent the historical data of the past T steps , spatiotemporal graph convolutional network model Will output the future T P Step-by-step prediction spatiotemporal data , d is the feature number of spatiotemporal features.

[0044] According to an embodiment of the present disclosure, the input data x and the predicted spatiotemporal data y are stacked into a tensor along the time dimension We use Represents the data of all nodes at the tth time step, using Represents the data of the uth node in all time steps. And the tensor T is normalized to improve the stability of learning, that is, for ,have . And map each data point in T to a random variable T. Specifically, the joint distribution of T can be obtained according to the probability density function in the Gaussian Markov Random Field (GMRF), as shown in the following formula (1).

[0045]

[0046] in, , is a symmetric positive definite parameter, is the parameter of the Gaussian Markov random field and is a non-negative parameter. The function Φ(T|W, θ) represents the potential flow direction of spatiotemporal data and can also be considered as the modeling of the spatiotemporal dynamic system, as shown in the following formula (2).

[0047]

[0048] in, is the potential energy matrix, which represents the mixed dependence of the variable T in the time and space dimensions. The specific representation is shown in formula (3).

[0049]

[0050] in, is a graph-like Laplacian operator, is the identity matrix of dimension N, is a regularization operator. It can be the regularized graph Laplacian , is the diagonalization operator. is a vectorization operator that flattens the first two dimensions of a tensor, i.e. . It is the matrix outer product, that is, the Kronecker product. A represents the relationship between nodes and can be an adjacency matrix.

[0051] According to the embodiments of the present disclosure, it can be considered that all values ​​of T are obtained by joint sampling from the random variable T through formula (1).

[0052] According to the embodiments of the present disclosure, spatiotemporal data prediction generally refers to learning the conditional distribution of future variable y relative to historical input data x=x, that is, y|x=x. The present disclosure incorporates this goal into the GMRF model, which can be parsed and represented by a combination of spatiotemporal operations, thereby introducing the Gaussian Markov Random Field (GMRF) as a generative model and exploring how the GMRF model that satisfies conditional expectations can lead to a label propagation algorithm based on a spatiotemporal graph, thereby better modeling the spatiotemporal data prediction context. Specifically, for any future time step, , predicted spatiotemporal data of all nodes The GMRF model is expected to satisfy the following formula (4) for the historical data x.

[0053]

[0054] in, is a scalar that represents the strength of the relationship between nodes. Characterizes the intensity of all nodes spreading their own data outward at time t', It represents the ability of a node to maintain its original characteristics when it is affected by other nodes at time t'. The larger the value, the less likely it is to be affected by other nodes. is a coefficient vector, which represents the importance or contribution of all past time to the result of the future step t. It can be regarded as a weight coefficient, which acts on the time dimension . Performs matrix multiplication of the tensor along its second dimension.

[0055] According to an embodiment of the present disclosure, the above equation (4) aggregates information in the spatial and temporal dimensions based on the input data x to generate a prediction. is equivalent to a graph convolution operation. In practice, due to Therefore, k can be fixed to a small value (e.g. k=2) to solve the problem of over-smoothing. On the other hand, the time operator It is used to model the relationship between the current prediction time step and the past T time steps. It is usually used as part of a time series model, such as a TCN (Temporal Convolutional Network) model, an RNN () model, or a Transformer model.

[0056] According to the embodiments of the present disclosure, based on the above GMRF model, the present disclosure finds that predicting future data only by input data as shown in formula (4) seems to ignore the spatiotemporal features in the future prediction data and only focus on the spatiotemporal features in the input data. Consider the future prediction data of node u at time step t, that is, the prediction label sub-data, expressed as , which is correlated with the future prediction data of other nodes, as shown in formula (5).

[0057]

[0058] Therefore, if it is known And use it as a condition for GMRF to predict the variable It can minimize the difference with the real label data The present disclosure finds that for any future time step , the data variable of node u at the future time step t About historical data x and data of other nodes in the current time step The condition is expected to satisfy the following formula (6).

[0059]

[0060] Among them, the basic prediction term, that is, the predicted label data, corresponds to formula (4), and the residual term corresponds to the difference between the basic prediction data and the true label data.

[0061]

[0062] Among them, in the smoothing coefficient term, is a scalar, , The smoothing coefficient represents the affinity between node u and the rest of the nodes.

[0063] According to an embodiment of the present disclosure, a GMRF model that satisfies conditional expectations is constructed to characterize the spatiotemporal graph prediction scenario, and through formula (4), it is found that only by using input data x to predict future data, that is, predicting label data, the influence between nodes is ignored, and based on this point, it is found that more accurate spatiotemporal data prediction can be achieved through the influence between input data and nodes, and based on this point, a spatiotemporal residual information propagation module is constructed.

[0064] According to the embodiments of the present disclosure, the target residual data and the predicted spatiotemporal data are fused to achieve correction of the predicted spatiotemporal data, thereby obtaining more accurate target prediction data.

[0065] According to an embodiment of the present disclosure, the spatiotemporal residual information propagation module includes: a prediction layer and a smoothing layer; inputting the target vector and the predicted label vector into the spatiotemporal residual information propagation module to obtain target residual data representing the prediction deviation of the predicted spatiotemporal data may include the following operations.

[0066] Based on the target vector and the predicted label vector, a deviation vector representing the temporal and spatial feature deviation between the target vector and the predicted label vector is obtained; the deviation vector is input into a prediction layer for predicting the prediction deviation of the predicted label vector to obtain an initial residual vector; the initial residual vector is input into a smoothing layer to obtain a target residual vector, wherein the smoothing layer is used to perform data correction on the initial residual vector based on the correlation relationship between multiple nodes; the target residual vector is inversely quantized to obtain target residual data.

[0067] According to the embodiments of the present disclosure, it can be known from formula (6) that the future prediction data can be corrected by the deviation between the future prediction data, i.e. the prediction label data and the real label data, which can also be called residual. However, in actual prediction, there is no real label, and it is impossible to know the deviation between the prediction label data and the real label. Therefore, the prediction deviation of the prediction label data can be predicted by setting a prediction layer in the spatiotemporal residual information propagation module. At the same time, the prediction deviation can be corrected by using the association relationship between nodes through the smoothing layer.

[0068] According to an embodiment of the present disclosure, the prediction layer can be obtained by a neural network, specifically, by a spatiotemporal graph convolutional network, and the prediction layer can have a structure similar to that of the spatiotemporal learning layer.

[0069] According to the embodiments of the present disclosure, for spatiotemporal data, the deviation between the predicted label data and the true label data is actually mainly caused by the difference in spatiotemporal characteristics between the input data, i.e., the spatiotemporal graph to be tested, and the predicted label data. Therefore, when modeling the residual term, the necessity of direct access to the true label data is eliminated by comparing the target vector and the predicted label data. Then, the deviation between the target vector and the predicted label vector can be determined and input into the prediction layer for learning, thereby obtaining the prediction deviation of the predicted label vector.

[0070] According to an embodiment of the present disclosure, the residual learning model also includes: a backtracking layer and a decoupling layer; based on the initial space-time vector and the predicted space-time vector, a target deviation vector representing the space-time feature deviation between the initial space-time vector and the predicted space-time vector is obtained, which may include the following operations.

[0071] The predicted label vector is input into the backtracking layer to obtain the processed predicted label vector, wherein the backtracking layer is used to map the predicted label vector to the same vector space as the target vector; the processed predicted label vector and the target vector are input into the decoupling layer to obtain the deviation vector, wherein the decoupling layer is used to determine the spatiotemporal feature deviation between the target vector and the predicted label vector.

[0072] According to the embodiments of the present disclosure, the specific implementation method of the backtracking layer is not limited, and it can be implemented by a multilayer perceptron (Multilayer Perceptron with Activation Functions, MLP) through a Gaussian Error Linear Unit (Gaussian Error Linear Unit, GELU) activation function, or it can be implemented by other methods.

[0073] According to an embodiment of the present disclosure, the process of mapping the predicted label vector to the same vector space as the target vector can be a process of adjusting the vector dimensions of the two to be consistent. Specifically, it can be characterized as: , where z h is the predicted label vector, T represents the time step, N represents the number of nodes, and d e Characterizes the number of channels.

[0074] According to an embodiment of the present disclosure, at the decoupling layer, subtracting the predicted label vector from the target vector can determine the temporal and spatial feature deviation between the two, that is, the deviation vector, and can filter out redundant temporal and spatial features.

[0075] According to an embodiment of the present disclosure, the process of obtaining the initial residual vector through the above process can be shown as the following formula (8).

[0076]

[0077] Among them, represents the initial residual vector, represents the prediction layer, represents the target vector, represents the backtracking layer.

[0078] According to an embodiment of the present disclosure, the smoothing layer includes I smoothing sub-layers, where I is a positive integer greater than 1; the I smoothing sub-layers are connected in sequence; each smoothing sub-layer includes a plurality of residual kernels; inputting the initial residual vector into the smoothing layer to obtain the target residual vector may include the following operations.

[0079] In the case of i = 1, input the initial residual vector into the first smoothing sub-layer to obtain the first smoothed residual vector, where the smoothing sub-layer is used to perform data correction on the initial residual vector by using the smoothing coefficients obtained from the plurality of residual kernels and the association relationships between the plurality of nodes; in the case of 1 < i ≤ I, input the (i - 1)-th initial residual vector into the i-th smoothing sub-layer to obtain the i-th smoothed residual vector, where in the case of i = I, the i-th smoothed residual vector is the target residual vector.

[0080] According to an embodiment of the present disclosure, the types of the plurality of residual kernels include: predefined kernels, diffusion kernels, adaptive kernels, and data-driven kernels.

[0081] According to an embodiment of the present disclosure, the smoothing process in each smoothing sub-layer is similar to the aggregation process based on residual kernels, i.e., graph kernels.

[0082] According to an embodiment of the present disclosure, the predefined kernel is constructed based on various prior information. For example: the geographical information of the nodes. For example, in a traffic scenario, when the observation object is a road monitoring point, by calculating the geographical distance matrix between each road monitoring point , in the manner of the following formulas (9) - (10), obtain the adjacency matrix kernel in the predefined kernel.

[0083]

[0084]

[0085] Among them, the degree matrix, which is composed of the diagonal elements of. is the indicator function, when the following condition holds, its value is 1, otherwise 0. is a hyperparameter used to filter extremely weak similarities to reduce the training burden. is the distance matrix the standard deviation of all elements inside. is the Hadamard product, representing the element-wise multiplication of matrices.

[0086] According to an embodiment of the present disclosure, for example, in an atmospheric scene, the observation object is an atmospheric detection station, and by calculating the distance between the atmospheric detection stations and relative altitude , according to the following formulas (11)~(12), the adjacency matrix kernel in the predefined kernel is obtained.

[0087]

[0088]

[0089] in, Used to filter out large altitude differences. The internal elements are .

[0090] According to an embodiment of the present disclosure, a diffusion kernel represents a diffusion process. Assuming that information is transmitted from one node to an adjacent node with a certain conversion probability, the diffusion kernel can capture the spatial and temporal characteristics of the information and consider the connection relationship between nodes.

[0091] According to an embodiment of the present disclosure, the adaptive core is embedded by two learnable nodes. Generation can capture more complex node features from the data, and the adaptive kernel can better focus on the global situation of the data from a temporal perspective.

[0092] According to an embodiment of the present disclosure, a data-driven kernel is generated by a complex neural network. For example, a Transformer is used to calculate the residual kernel. The data-driven kernel can capture complex structures in the data and complex relationships between data, such as nonlinear relationships and interactions. Furthermore, the data-driven kernel can better focus on the local situation of the data from a temporal perspective.

[0093] According to an embodiment of the present disclosure, when K residual cores are deployed in each residual sub-layer, the residual coefficients can be obtained as shown in the following formulas (13) to (15).

[0094]

[0095]

[0096]

[0097] in, represents the i-th residual kernel, The element size of represents the intensity of residual diffusion in the global spatiotemporal graph corresponding to a specific node. It is the determining factor of the overall magnitude of the residual effect.

[0098] According to an embodiment of the present disclosure, The sign of the element determines whether the correlation between the corresponding specific nodes is positive or negative, thereby avoiding the weak regression of the overly strong homogeneity assumption for abnormal data nodes.

[0099] According to an embodiment of the present disclosure, under the combination of multiple residual kernels, the residual propagation in each smoothing sub-layer is equivalent to a linear graph convolution operation with residual skip connection.

[0100] According to an embodiment of the present disclosure, the smoothing process of the overall smoothing layer for the initial residual vector may be as shown in the following formula (16).

[0101]

[0102] in, represents the target residual vector, represents the smoothing layer, and L represents the number of smoothing sublayers.

[0103] According to the embodiments of the present disclosure, in each smoothing sub-layer, multiple residual kernels work together to focus on different features of the input data from different angles, thereby obtaining a more comprehensive and accurate smoothing coefficient, and making the result of data correction of the initial residual vector through the smoothing coefficient more accurate.

[0104] According to an embodiment of the present disclosure, multiple residual kernels include a predefined kernel that characterizes the association relationship between multiple nodes; the smoothing coefficient includes: a smoothing sub-coefficient between multiple nodes; the initial residual vector includes the initial residual sub-vector of each of the multiple nodes; and data correction of the initial residual vector using the smoothing coefficient obtained by the multiple residual kernels and the association relationship between the multiple nodes may include the following operations.

[0105] For each node, a target node associated with the node is determined from a predefined kernel; a smoothing sub-coefficient between the target node and the node is determined from a smoothing coefficient; an initial residual sub-vector of the target node is determined from an initial residual vector; and based on the smoothing sub-coefficient and the initial residual sub-vector of the target node, the target residual sub-data of the node is obtained.

[0106] According to an embodiment of the present disclosure, in each smoothing sub-layer, the initial residual sub-vector of the node is corrected by other nodes, namely the target nodes, that have a specific association relationship with the node. Specifically, the target residual sub-data of the node can be obtained through the smoothing sub-coefficient between the target node and the node and the initial residual sub-vector of the target node. For example, the smoothing sub-coefficients of the target node 1, the target node 2 and the node are 0.1 and 0.2 respectively, and the target residual sub-data of the target node 1 and the target node 2 are a and b, then the target residual sub-data of the node is 0.1a+0.2b.

[0107] According to an embodiment of the present disclosure, by correcting the initial residual sub-data of the node by other nodes except the node, the goal expected in formula (6) can be achieved, that is, combining the predicted label sub-data of other nodes except the current node to minimize the predicted difference between the current node and the true label data, thereby achieving more accurate prediction.

[0108] According to an embodiment of the present disclosure, the method may include the following operations.

[0109] Based on the position information of multiple observation objects included in the spatiotemporal data to be tested, the time series data corresponding to each of the multiple observation objects, and the association relationship between each of the multiple observation objects, a spatiotemporal graph to be tested is obtained, wherein the nodes are determined by the position information of the observation objects and the time series data corresponding to each of the observation objects, and the edges are determined by the association relationship between each of the multiple observation objects.

[0110] According to an embodiment of the present disclosure, the spatiotemporal data to be measured includes multiple observation objects, respective position information of the multiple observation objects, time series data corresponding to each of the multiple observation objects, etc. The association relationship between each of the multiple observation objects may be included in the spatiotemporal data to be measured, or may be obtained from a database or memory and then added to the spatiotemporal data to be measured.

[0111] According to an embodiment of the present disclosure, the spatiotemporal data to be tested may be constructed into a spatiotemporal graph to be tested through GMRF.

[0112] According to an embodiment of the present disclosure, by converting the spatiotemporal data to be tested into a spatiotemporal graph to be tested, the subsequent spatiotemporal graph convolutional network and spatiotemporal residual information propagation module can better learn the spatiotemporal features included in the input data, thereby outputting more accurate results.

[0113] According to an embodiment of the present disclosure, the following operations may also be included.

[0114] When the observation object is a road monitoring point, the time series data includes the traffic data of the road monitoring point within the target time period, and the association relationship between the multiple observation objects includes the positional relationship between the multiple road observation points; when the observation object is an atmospheric detection station, the time series data includes the atmospheric data of the atmospheric detection station within the target time period, and the association relationship between the multiple observation objects includes the positional relationship between the multiple atmospheric detection stations.

[0115] According to the embodiments of the present disclosure, in different scenarios, there are differences in the observed objects, time series data, and the association relationships between multiple observed objects.

[0116] Figure 2 The model architecture diagram of the spatiotemporal data prediction method according to an embodiment of the present disclosure is schematically shown.

[0117] like Figure 2 As shown, the method includes converting the spatiotemporal data 211 to be tested into a spatiotemporal graph 212 to be tested, and inputting the spatiotemporal graph 212 to be tested into a spatiotemporal graph convolutional network 220, wherein the spatiotemporal graph convolutional network 220 may include an encoder 221, a spatiotemporal learning layer 222 and a decoder 225.

[0118] According to an embodiment of the present disclosure, the space-time graph to be tested 212 is input to the encoder 221 for vectorization, so as to map the space-time graph to be tested to a high-dimensional feature space, and obtain a target vector 222. Specifically, it can be represented in the following way: , where x is the space-time graph 212 to be tested, is the target vector 222. The encoder 221 can be obtained by combining enhancement strategies such as node embedding technology.

[0119] According to an embodiment of the present disclosure, the target vector 222 is input to the spatiotemporal learning layer 222 for prediction to obtain a predicted label vector 224. Specifically, the spatiotemporal learning layer , is the number of channels, To predict the label vector 224 , the spatiotemporal learning layer is used to capture the spatiotemporal features of the target vector 222 and generate the predicted label vector 224 .

[0120] According to an embodiment of the present disclosure, the predicted label vector data 224 is inversely quantized in the decoder 225 to obtain the predicted spatiotemporal data 226. Specifically, the decoder Decode the predicted spatiotemporal data 226 to generate That is, predicting spatiotemporal data 226.

[0121] According to an embodiment of the present disclosure, the target vector 222 and the prediction label vector 224 are input into the spatiotemporal residual information propagation module 230, and the target residual data 237 is output. Specifically, the residual prediction model 220 may include a traceback layer 231, a decoupling layer 233, a prediction layer 234, a smoothing layer 235, and a decoder 236. By inputting the prediction label vector 224 into the traceback layer 231, a target spatiotemporal vector 232 in the same vector space as the target vector 222 is obtained, and the target spatiotemporal vector 232 and the target vector 222 are input into the decoupling layer 222 to obtain a deviation vector, and the deviation vector is input into the prediction layer 234 to obtain an initial residual vector, and the initial residual vector is input into the smoothing layer for smoothing operation to obtain the target residual vector, and the target residual vector is decoded by the decoder 236 to obtain the target residual data 237. Specifically, the decoder , used to generate the correction term .

[0122] According to an embodiment of the present disclosure, the decoder 236 in the spatiotemporal residual information propagation module 230 is not limited and may be a multi-layer perceptron (MLP) layer of a Gaussian error linear unit (GELU) activation function.

[0123] According to an embodiment of the present disclosure, the target residual data 237 and the initial prediction data 226 are fused so that the target residual data 237 corrects the prediction label data 226 to obtain the target prediction data 241. Specifically, this correction item, i.e., the target prediction data 241, is added to the basic prediction, i.e., the prediction label data , to obtain the final revised prediction, i.e. the target prediction data .

[0124] According to an embodiment of the present disclosure, by integrating the predicted label vector into the spatiotemporal graph learning, the predicted label vector can be propagated in the spatiotemporal graph learning, thereby achieving the beneficial effect of enhancing the performance of the spatiotemporal graph convolutional network. Since this propagation strategy is independent of the spatiotemporal graph convolutional network, it ensures that most spatiotemporal graph convolutional networks can benefit from this method.

[0125] According to the embodiments of the present disclosure, the residual effectively incorporates future prediction data, i.e., prediction label vectors, into spatiotemporal learning through the spatiotemporal residual information propagation module, i.e., STRIP. First, it decouples the residual components from the input target vector and prediction label vector. Then, it smoothes these residuals by incorporating spatiotemporal correlation, i.e., a smoothing layer. Finally, the prediction results are corrected using the target residual data improved by this module, thereby improving the accuracy of the model.

[0126] According to an embodiment of the present disclosure, the training method of the spatiotemporal graph convolutional network and the spatiotemporal residual information propagation module may include one of the following methods.

[0127] The initial spatiotemporal graph convolutional network and the initial spatiotemporal residual information propagation module are jointly trained to obtain the spatiotemporal graph convolutional network and the spatiotemporal residual information propagation module, wherein the initial spatiotemporal graph convolutional network and the initial spatiotemporal residual information propagation module are both untrained initial models; or, when the spatiotemporal graph convolutional network has been obtained, the model parameters of the spatiotemporal graph convolutional network are fixed, and the residual prediction model is trained based on the spatiotemporal graph convolutional network to obtain a trained residual prediction model.

[0128] According to an embodiment of the present disclosure, the initial spatiotemporal graph convolutional network and the initial spatiotemporal residual information propagation module are jointly trained to obtain the spatiotemporal graph convolutional network and the spatiotemporal residual information propagation module. The specific steps may include the following contents.

[0129] The sample spatiotemporal graph is input into the initial spatiotemporal graph convolutional network to obtain the sample target vector, the sample prediction label vector of the sample vector and the sample prediction spatiotemporal data corresponding to the sample prediction label vector; the sample target vector and the sample prediction label vector are input into the initial spatiotemporal residual information propagation module to obtain the sample target residual data; the sample prediction spatiotemporal data is corrected based on the sample target residual data to obtain the sample target prediction data; the sample target prediction data and the true label data are input into the loss function to obtain the loss value, and the parameters of the initial spatiotemporal graph convolutional network and the initial spatiotemporal residual information propagation module are adjusted based on the loss value to obtain the spatiotemporal graph convolutional network and the spatiotemporal residual information propagation module.

[0130] According to the embodiments of the present disclosure, there is no limitation on the training methods of the spatiotemporal graph convolutional network and the spatiotemporal residual information propagation module. When the spatiotemporal graph convolutional network cannot adjust parameters, the spatiotemporal residual information propagation module can be combined with the spatiotemporal graph convolutional network, and the spatiotemporal residual information propagation module can be trained based on the spatiotemporal graph convolutional network.

[0131] According to the embodiments of the present disclosure, experiments were conducted on datasets in the two fields of atmosphere and traffic. Table 1 shows the spatiotemporal graph data features of each dataset.

[0132] Table 1

[0133]

[0134] According to an embodiment of the present disclosure, the SD and GBA traffic datasets are obtained from the recent traffic benchmark dataset LargeST, which records the traffic speed data from the California Department of Transportation (CalTrans) Performance Measurement System (PeMS) highway system. The detailed data is shown in Table 2.

[0135] Table 2

[0136]

[0137] According to an embodiment of the present disclosure, the spatiotemporal graph convolutional network can be implemented by the following types of neural networks, including: Long Short-Term Memory (LSTM), Spatio-Temporal Graph Convolutional Network (STGCN), Spatio-Temporal Temporal Network (STTN), Advanced Spatio-Temporal Graph Convolutional Network (ASTGCN), Attention-based Graph Convolutional Recurrent Network (AGCRN), Dynamic Graph Convolutional Recurrent Network (DGCRN), Differential Graph Convolutional Recurrent Network (DDGCRN), STID, STAEFormer, etc.

[0138] According to an embodiment of the present disclosure, Tables 3 and 4 show the experimental performance of the STRIP module combined with various types of spatiotemporal graph convolutional networks on traffic datasets.

[0139] Table 3

[0140]

[0141] Table 4

[0142]

[0143] According to the embodiments of the present disclosure, the methods in Tables 3 and 4, that is, the neural network model for implementing the spatiotemporal graph convolutional network, are recorded under Method, - recorded under STRIP represents the case where the spatiotemporal residual information propagation module, that is, the STRIP module, is not combined, JT represents the case where the spatiotemporal graph convolutional network and the STRIP module are jointly trained, and FT represents the case where the training is carried out by fine-tuning, that is, the parameters of the spatiotemporal graph convolutional network are fixed and only the STRIP module is trained. Horizon 3 represents the moment in the future time, and Horizon 3 refers to the third moment. The following values ​​correspond to the performance indicators of the model at this moment. The same is true for Horizon 6 and Horizon 9. Specifically, when the data sampling frequency is 15 minutes, Horizon 3 represents the 45th minute in the future. Average relative improvement refers to the average relative improvement, and MAE (Mean Absolute Error), RMSE (Root Mean Squared Error) and MAPE (Mean Absolute Percentage Error) are measurement indicators.

[0144] According to the embodiments of the present disclosure, in Tables 3 and 4, when the spatiotemporal graph convolutional network is combined with the STRIP module of the present disclosure, they all show performance improvements. This improvement is particularly evident in some networks with poor performance, such as ASTGCN, which has an average improvement of 15% on the SD dataset. Even for some competitive benchmark models such as STID and AGCRN, our module can still bring significant performance improvements. For example, the average MAPE of STID in the GBA dataset can be increased by 7.0%.

[0145] According to the embodiments of the present disclosure, when comparing the two training strategies of the STRIP module, it can be clearly seen that the joint training method is generally superior to the training method when the parameters of the spatiotemporal graph convolutional network model are fixed in various cases. This advantage can be attributed to the enhanced adaptability provided by the joint training, which enables more effective capture of complex spatiotemporal correlations. In summary, the above experimental results show that the STRIP module significantly improves the prediction accuracy of the spatiotemporal prediction model in traffic scenarios.

[0146] Figure 3 (a) schematically shows a first schematic diagram of the implementation result of the spatiotemporal data prediction method according to an embodiment of the present disclosure; Figure 3 (b) schematically shows a second schematic diagram of the implementation result of the spatiotemporal data prediction method according to an embodiment of the present disclosure; Figure 3 (c) schematically shows a third schematic diagram of the implementation result of the spatiotemporal data prediction method according to an embodiment of the present disclosure.

[0147] As shown in Figure 3 (a) to Figure 3 (c), the prediction cases in Table 3 to Table 3 are visualized. In 311, 321, and 331, Ground truth represents the real label data, and the real label data is obtained through the sample data set. With STRIP+JT represents the target prediction data output by the model obtained by combining the spatiotemporal graph convolutional network with the STRIP module and training through joint training. With STRIP+FT uses the target prediction data output by the model obtained by combining the spatiotemporal graph convolutional network with the STRIP module and fine-tuning training. Without STRIP represents only the predicted label data output by the spatiotemporal graph convolutional network. In 312, 322, and 332, Base prediction error represents the difference between the real label data and the predicted label data, Risidual in STRIP+JT represents the difference between the target prediction data corresponding to With STRIP+JT and the real label data, and Risidual in STRIP+FT represents the difference between the target prediction data corresponding to With STRIP+FT and the real label data.

[0148] According to the embodiments of the present disclosure, observing FIG3 (a) to FIG3 (c), the STRIP module can help the model learn abnormal signals well, which is called spatiotemporal deviation. The figure shows the predicted label data y base and real label data The error between ), and the target residual data learned by STRIP (i.e. ). The improvement is due to the fact that the correction term effectively compensates for the gap in the basic forecast.

[0149] Figure 4 (a) schematically shows a first visual graph of the convergence status when the spatiotemporal graph convolutional network and the spatiotemporal residual information propagation module are trained using the first data set according to an embodiment of the present disclosure; Figure 4 (b) schematically shows a second visual graph of the convergence status when the spatiotemporal graph convolutional network and the spatiotemporal residual information propagation module are trained using the first data set according to an embodiment of the present disclosure; Figure 4 (c) schematically shows a third visual graph of the convergence status when the spatiotemporal graph convolutional network and the spatiotemporal residual information propagation module are trained using the first data set according to an embodiment of the present disclosure; Figure 4 (d) schematically shows a fourth visual graph of the convergence status when the spatiotemporal graph convolutional network and the spatiotemporal residual information propagation module are trained using the first data set according to an embodiment of the present disclosure.

[0150] As shown in Figure 4 (a) to Figure 4 (d), the prediction cases of LSTM, ASTGCN, and STGCN are visualized. The StartFine-tuning representation begins fine-tuning. From 411 to 443, it can be seen that under different indicators, the STRIP module can speed up the overall convergence speed, especially when using a joint training strategy. However, since STRIP requires an additional residual learning step using a spatiotemporal model, it does increase the training time. Specifically, this additional time is approximately equivalent to the time required for one training iteration. However, considering that the spatiotemporal data spans several years, this increased training time is negligible.

[0151] According to an embodiment of the present disclosure, the KnowAir dataset records 4 years of PM2.5 characteristics from 184 air monitoring stations and is further divided into three sub-datasets. The detailed information of the dataset is shown in Table 5.

[0152] Table 5

[0153]

[0154] According to an embodiment of the present disclosure, the effectiveness of the proposed module is analyzed on the KnowAir dataset in the atmosphere field. In addition to the above-mentioned spatiotemporal prediction models, specialized models that perform well in atmospheric prediction tasks are additionally supplemented, including GC-LSTM, nodesFC-GRU, and PM2.5GNN. In addition, in order to conduct a more comprehensive evaluation, the present invention introduces several indicators widely used in the field of atmospheric prediction, including the critical success index (CSI), the probability of detection (POD), and the false alarm rate (FAR). Higher values ​​of the first two indicators indicate better performance.

[0155] Table 6

[0156]

[0157] Table 6

[0158]

[0159] Table 7

[0160]

[0161] Table 7

[0162]

[0163] Table 8

[0164]

[0165] Table 8

[0166]

[0167] According to the embodiments of the present disclosure, it can be seen from Tables 6 to 8 that the STRIP module still performs best on the KnowAir dataset, even surpassing several specialized atmospheric prediction models. This is because the fundamental challenge in spatiotemporal atmospheric prediction and traffic tasks is modeling spatiotemporal correlations, and STRIP obviously does a better job. The STRIP module of the present disclosure can significantly help the spatiotemporal graph model predict future atmospheric data.

[0168] Figure 6 A visual diagram schematically shows the convergence status when the spatiotemporal graph convolutional network and the spatiotemporal residual information propagation module are trained using the second data set according to an embodiment of the present disclosure.

[0169] like Figure 6 As shown, at 610~660, by visualizing the training process of several spatiotemporal graph convolutional networks using STRIP, it can be clearly seen that STRIP accelerates the convergence of the model, especially when using a joint training strategy. However, since STRIP requires an additional residual learning step using a spatiotemporal model, it does increase the training time. Specifically, this additional time is approximately equivalent to the time required for one training iteration. However, considering that the dataset of this implementation example still spans several years, this increased training time is negligible. At the same time, the conclusions in the transportation field are similar, and it is still found in this implementation example that STRIP can speed up the convergence.

[0170] Based on the above-mentioned spatiotemporal data prediction method, the present disclosure also provides a spatiotemporal data prediction device. Figure 7 The device is described in detail.

[0171] Figure 7 The structural block diagram of the spatiotemporal data prediction device according to an embodiment of the present disclosure is schematically shown.

[0172] like Figure 7 As shown, the spatiotemporal data prediction device 700 of this embodiment includes an initial prediction module 710 , a residual prediction module 720 and a correction module 730 .

[0173] The initial prediction module 710 is used to input the spatiotemporal graph to be tested into the spatiotemporal graph convolutional network to obtain a target vector representing the spatiotemporal graph to be tested, a predicted label vector of the target vector, and predicted spatiotemporal data corresponding to the predicted label vector, wherein the predicted label vector is obtained by predicting the target vector, and the predicted spatiotemporal data is obtained by inverse quantization of the predicted label vector, and the spatiotemporal graph to be tested includes multiple nodes and edges for characterizing the relationship between the multiple nodes.

[0174] The residual prediction module 720 is used to input the target vector and the predicted label vector into the spatiotemporal residual information propagation module to obtain the target residual data representing the prediction deviation of the predicted spatiotemporal data, wherein the spatiotemporal residual information propagation module is determined based on the Gauss-Markov hypothesis model, and the target residual data includes target residual sub-data of each of the multiple nodes. The spatiotemporal residual information propagation module is used to determine the target residual sub-data of each of the multiple nodes based on the deviation vector between the target vector and the predicted label vector and the correlation relationship between the multiple nodes.

[0175] The correction module 730 is used to correct the predicted spatiotemporal data based on the target residual data to obtain the target predicted data for the spatiotemporal graph to be tested.

[0176] According to an embodiment of the present disclosure, any multiple modules of the initial prediction module 710, the residual prediction module 720 and the correction module 730 can be combined into one module for implementation, or any one of the modules can be split into multiple modules. Alternatively, at least part of the functions of one or more of these modules can be combined with at least part of the functions of other modules and implemented in one module. According to an embodiment of the present disclosure, at least one of the initial prediction module 710, the residual prediction module 720 and the correction module 730 can be at least partially implemented as a hardware circuit, such as a field programmable gate array (FPGA), a programmable logic array (PLA), a system on a chip, a system on a substrate, a system on a package, an application specific integrated circuit (ASIC), or can be implemented by hardware or firmware such as any other reasonable way of integrating or packaging the circuit, or implemented in any one of the three implementation methods of software, hardware and firmware or in any appropriate combination of any of them. Alternatively, at least one of the initial prediction module 710, the residual prediction module 720 and the correction module 730 can be at least partially implemented as a computer program module, and when the computer program module is run, the corresponding function can be performed.

[0177] FIG8 schematically shows a block diagram of an electronic device suitable for implementing a spatiotemporal data prediction method according to an embodiment of the present disclosure.

[0178] As shown in FIG8 , the electronic device 800 according to an embodiment of the present disclosure includes a processor 801, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 802 or a program loaded from a storage part 809 to a random access memory (RAM) 803. The processor 801 may include, for example, a general-purpose microprocessor (e.g., a CPU), an instruction set processor and / or a related chipset and / or a dedicated microprocessor (e.g., an application-specific integrated circuit (ASIC)), etc. The processor 801 may also include an onboard memory for caching purposes. The processor 801 may include a single processing unit or multiple processing units for performing different actions of the method flow according to an embodiment of the present disclosure.

[0179] In RAM 803, various programs and data required for the operation of electronic device 800 are stored. Processor 801, ROM 802 and RAM 803 are connected to each other via bus 804. Processor 801 performs various operations of the method flow according to the embodiment of the present disclosure by executing the program in ROM 802 and / or RAM 803. It should be noted that the program can also be stored in one or more memories other than ROM 802 and RAM 803. Processor 801 can also perform various operations of the method flow according to the embodiment of the present disclosure by executing the program stored in the one or more memories.

[0180] According to an embodiment of the present disclosure, the electronic device 800 may further include an input / output (I / O) interface 805, which is also connected to the bus 804. The electronic device 800 may further include one or more of the following components connected to the input / output (I / O) interface 805: an input portion 806 including a keyboard, a mouse, etc.; an output portion 807 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage portion 808 including a hard disk, etc.; and a communication portion 809 including a network interface card such as a LAN card, a modem, etc. The communication portion 809 performs communication processing via a network such as the Internet. A drive 810 is also connected to the input / output (I / O) interface 805 as needed. A removable medium 811, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 810 as needed, so that a computer program read therefrom is installed into the storage portion 809 as needed.

[0181] The flow charts and block diagrams in the accompanying drawings illustrate the possible architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each box in the flow chart or block diagram can represent a module, a program segment, or a part of a code, and the above-mentioned module, program segment, or a part of a code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flow chart, and the combination of the boxes in the block diagram or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0182] It will be appreciated by those skilled in the art that the features described in the various embodiments of the present disclosure may be combined and / or combined in a variety of ways, even if such combinations or combinations are not explicitly described in the present disclosure. In particular, without departing from the spirit and teachings of the present disclosure, the features described in the various embodiments of the present disclosure may be combined and / or combined in a variety of ways. All of these combinations and / or combinations fall within the scope of the present disclosure.

[0183] The embodiments of the present disclosure are described above. However, these embodiments are only for illustrative purposes and are not intended to limit the scope of the present disclosure. Although the embodiments are described above, this does not mean that the measures in the various embodiments cannot be used in combination to advantage. Without departing from the scope of the present disclosure, those skilled in the art may make a variety of substitutions and modifications, which should all fall within the scope of the present disclosure.

Claims

1. A method for predicting spatiotemporal data, characterized in that: The method includes: Inputting the spatio-temporal graph to be measured into a spatio-temporal graph convolutional network to obtain a target vector representing the spatio-temporal graph to be measured, a predicted label vector of the target vector, and predicted spatio-temporal data corresponding to the predicted label vector, where the predicted label vector is obtained by predicting the target vector, the predicted spatio-temporal data is obtained by inverse quantization of the predicted label vector, and the spatio-temporal graph to be measured includes multiple nodes and edges for characterizing the association relationships between the multiple nodes; Inputting the target vector and the predicted label vector into the spatio-temporal residual information propagation module to obtain target residual data characterizing the prediction deviation of the predicted spatio-temporal data; Based on the target residual data, correcting the predicted spatio-temporal data to obtain target prediction data for the spatio-temporal graph to be measured; Among them, the spatio-temporal residual information propagation module is determined based on a Gaussian Markov hypothesis model, the target residual data includes target residual sub-data of each of the multiple nodes, and the spatio-temporal residual information propagation module is used to determine the target residual sub-data of each of the multiple nodes based on the deviation vector between the target vector and the predicted label vector and the association relationships between the multiple nodes themselves.

2. The method according to claim 1, characterized in that The spatio-temporal residual information propagation module includes: a prediction layer and a smoothing layer; The step of inputting the target vector and the predicted label vector into the spatio-temporal residual information propagation module to obtain target residual data characterizing the prediction deviation of the predicted spatio-temporal data includes: Based on the target vector and the predicted label vector, obtaining a deviation vector characterizing the spatio-temporal feature deviation between the target vector and the predicted label vector; Inputting the deviation vector into the prediction layer for predicting the prediction deviation of the predicted label vector to obtain an initial residual vector; Inputting the initial residual vector into the smoothing layer to obtain the target residual vector, where the smoothing layer is used to perform data correction on the initial residual vector based on the association relationships between the multiple nodes themselves; Performing inverse quantization on the target residual vector to obtain the target residual data.

3. The method according to claim 2, characterized in that The smoothing layer includes I smoothing sub-layers, where I is a positive integer greater than 1; the I smoothing sub-layers are connected in sequence; Each of the smoothing sub-layers includes multiple residual kernels; The step of inputting the initial residual vector into the smoothing layer to obtain the target residual vector includes: When i = 1, Inputting the initial residual vector into the first smoothing sub-layer to obtain a first smoothed residual vector, where the smoothing sub-layer is used to perform data correction on the initial residual vector by using the smoothing coefficients obtained from the multiple residual kernels and the association relationships between the multiple nodes themselves; When 1 < i ≤ I, Inputting the (i - 1)-th initial residual vector into the i-th smoothing sub-layer to obtain the i-th smoothed residual vector, where when i = I, the i-th smoothed residual vector is the target residual vector.

4. The method according to claim 3, characterized in that The plurality of residual kernels include predefined kernels that characterize the association relationship between the plurality of nodes; the smoothing coefficient includes: smoothing sub-coefficients between the plurality of nodes; the initial residual vector includes initial residual sub-vectors of the plurality of nodes; The method of performing data correction on the initial residual vector by using the smoothing coefficients obtained by the plurality of residual kernels and the association relationship between the plurality of nodes comprises: For each of the nodes, Determine a target node having an association relationship with the node from the predefined core; Determining a smoothing sub-coefficient between the target node and the node from the smoothing coefficient; Determine an initial residual subvector of the target node from the initial residual vector; Based on the smoothing sub-coefficient and the initial residual sub-vector of the target node, the target residual sub-data of the node is obtained.

5. The method according to claim 2, characterized in that: The residual learning model also includes: a backtracking layer and a decoupling layer; The step of obtaining a target deviation vector representing a deviation of spatiotemporal features between the initial spatiotemporal vector and the predicted spatiotemporal vector based on the initial spatiotemporal vector and the predicted spatiotemporal vector includes: Inputting the predicted label vector into the backtracking layer to obtain a processed predicted label vector, wherein the backtracking layer is used to map the predicted label vector to the same vector space as the target vector; The processed predicted label vector and the target vector are input into the decoupling layer to obtain the deviation vector, wherein the decoupling layer is used to determine the spatiotemporal feature deviation between the target vector and the predicted label vector.

6. The method according to claim 1, characterized in that The method comprises: Based on the position information of multiple observation objects included in the spatiotemporal data to be tested, the time series data corresponding to each of the multiple observation objects, and the association relationship between each of the multiple observation objects, a spatiotemporal graph to be tested is obtained, wherein the nodes are determined by the position information of the observation objects and the time series data corresponding to each of the observation objects, and the edges are determined by the association relationship between each of the multiple observation objects.

7. The method according to claim 6, characterized in that The method further comprises: In the case where the observation object is a road monitoring point, the time series data includes flow data of the road monitoring point in a target time period, and the association relationship between each of the plurality of observation objects includes a positional relationship between the plurality of road observation points; In the case where the observation object is an atmospheric monitoring station, the time series data includes atmospheric data of the atmospheric monitoring station within a target time period, and the association relationship between the plurality of observation objects includes a positional relationship between the plurality of atmospheric monitoring stations.

8. The method according to claim 1, characterized in that The training method of the spatiotemporal graph convolutional network and the spatiotemporal residual information propagation module includes: Jointly training an initial spatiotemporal graph convolutional network and an initial spatiotemporal residual information propagation module to obtain the spatiotemporal graph convolutional network and the spatiotemporal residual information propagation module, wherein both the initial spatiotemporal graph convolutional network and the initial spatiotemporal residual information propagation module are untrained initial models; Or, when the spatiotemporal graph convolutional network has been obtained, the model parameters of the spatiotemporal graph convolutional network are fixed, and the residual prediction model is trained based on the spatiotemporal graph convolutional network to obtain the trained residual prediction model.

9. A spatiotemporal data prediction device, characterized in that: The device comprises: An initial prediction module is used to input the space-time graph to be tested into the space-time graph convolutional network to obtain a target vector for representing the space-time graph to be tested, a predicted label vector of the target vector, and predicted space-time data corresponding to the predicted label vector, wherein the predicted label vector is obtained by predicting the target vector, and the predicted space-time data is obtained by inverse quantization of the predicted label vector, and the space-time graph to be tested includes multiple nodes and edges for characterizing the association relationship between the multiple nodes; A residual prediction module, used for inputting the target vector and the predicted label vector into the spatiotemporal residual information propagation module to obtain target residual data representing the prediction deviation of the predicted spatiotemporal data, wherein the spatiotemporal residual information propagation module is determined based on the Gauss-Markov hypothesis model, the target residual data includes target residual sub-data of each of the multiple nodes, and the spatiotemporal residual information propagation module is used for determining the target residual sub-data of each of the multiple nodes based on the deviation vector between the target vector and the predicted label vector and the association relationship between the multiple nodes; A correction module is used to correct the predicted spatiotemporal data based on the target residual data to obtain target predicted data for the spatiotemporal graph to be tested.

10. An electronic device comprising: one or more processors; a memory for storing one or more computer programs, It is characterized in that the one or more processors execute the one or more computer programs to implement the steps of the method according to any one of claims 1 to 8.

Images