Efficient parameter sharing city spatio-temporal data prediction method

Abstract

The invention relates to the technical field of data processing, in particular to an efficient parameter sharing city spatio-temporal data prediction method, which comprises the following steps of: defining a parameter sharing time expansion convolution operator and a parameter sharing graph diffusion convolution operator; a parameter sharing time expansion convolution operator and a parameter sharing graph diffusion convolution operator can effectively and quickly capture the space-time dependency relationship in the data under the condition that the model calculation time and the model parameter scale are not remarkably increased; a mathematical logic relationship among the hyper-parameters is established, the search space of the hyper-parameters is reduced, the parameter adjustment difficulty of the hyper-parameters is reduced, the trial and error time and resource investment are reduced, and the excellent performance of the PSDCNN model in a space-time prediction task is ensured. According to the PSDCNN model provided by the invention, the space-time prediction task can be accurately completed under the condition that the model calculation time / model parameter scale is not remarkably increased. The PSDCNN model has a prediction accuracy that exceeds or approximates a baseline method.

Description

Technical Field

[0001] The present invention relates to the technical field of data processing, and in particular to an efficient parameter-sharing urban spatiotemporal data prediction method. Background Art

[0002] The spatiotemporal prediction model is an important supporting technology for real-time urban monitoring and management, and is crucial to improving urban operation efficiency and the quality of life of residents. At present, the spatiotemporal prediction model has been widely used in practical applications such as intelligent transportation, weather forecasting, and earthquake warning. For example, in the intelligent transportation system, timely prediction of future traffic conditions can help urban traffic management departments balance traffic flow in the traffic network and alleviate urban traffic congestion. In the weather forecast system, predicting future rainfall in advance can help the meteorological department improve its disaster prevention and mitigation capabilities and reduce the negative impact of rainfall on the city.

[0003] Existing spatiotemporal prediction models can be roughly divided into statistical learning models, machine learning models, and deep learning models. Compared with statistical learning models and machine learning models, deep learning models have become the mainstream spatiotemporal prediction models by virtue of their ability to capture complex dependencies. In recent years, relevant scholars have proposed a variety of deep learning models to improve the prediction accuracy of spatiotemporal prediction tasks. From the early recurrent neural network model to the recent spatiotemporal graph neural network, the prediction accuracy of spatiotemporal prediction models has been continuously updated, and their performance in various fields has become increasingly outstanding. However, with the improvement of prediction accuracy, the current spatiotemporal prediction models have become more and more complex, bringing new challenges to spatiotemporal prediction tasks. Overly complex prediction models not only increase the difficulty of model implementation and parameter adjustment, but also waste a lot of computing resources and storage resources. In real scenarios, accurate, efficient and lightweight spatiotemporal prediction models have always been the focus of academic attention, but most existing prediction models still find it difficult to strike a balance between model prediction accuracy and model lightweightness.

[0004] In response to the above challenges, this application aims to obtain acceptable prediction accuracy without significantly increasing the model calculation time / model parameter scale. At present, some scholars have made some explorations on model lightweighting and established a few lightweight spatiotemporal prediction models, but there are still some shortcomings. Specifically, the existing models focus more on improving computational efficiency but ignore the control of parameter scale. Secondly, the existing models usually have more hyperparameters, which makes the parameter adjustment process complicated and time-consuming.

[0005] In view of this, this application proposes a novel parameter-shared dilated convolutional network (PSDCN) for spatiotemporal prediction tasks. Summary of the invention

[0006] In order to make up for the deficiencies in the prior art, the present invention provides an efficient parameter-sharing urban spatiotemporal data prediction method.

[0007] An efficient parameter-sharing urban spatiotemporal data prediction method is characterized by comprising the following steps:

[0008] S1, preparation stage, building the PSDCN model;

[0009] S2, defining a parameter-sharing time dilation convolution operator and a parameter-sharing graph diffusion convolution operator, wherein the parameter-sharing time dilation convolution operator is a PSTDC operator, and the parameter-sharing graph diffusion convolution operator is a PSGDC operator; the parameter-sharing time dilation convolution operator and the parameter-sharing graph diffusion convolution operator can effectively and quickly capture the spatiotemporal dependency in the data without significantly increasing the model calculation time and the model parameter scale;

[0010] S3, establishes the mathematical logic relationship between hyperparameters, narrows the search space of hyperparameters, reduces the difficulty of hyperparameter adjustment, reduces the time and resource investment of trial and error, and ensures the excellent performance of the PSDCN model in spatiotemporal prediction tasks;

[0011] S4, three real spatiotemporal datasets are used to evaluate the performance of the PSDCN model. The three real spatiotemporal datasets are PM 2.5 data set, traffic data set, and temperature data set. The performance of the PSDCN model includes prediction accuracy, operation efficiency, and the number of model parameters.

[0012] Furthermore, in order to better implement the present invention, the PSDCN model in S1 is constructed based on a graph structure. In the graph structure G = (V, A), each monitoring site in the study area can be abstracted as a node v i ∈V, monitoring site v i With v j The relationship between them can be abstracted as edge A ij ∈A, monitoring site v i The spatiotemporal data collected within the time window t can be recorded as Generally speaking, the spatiotemporal data collected by all monitoring stations in all time windows form a matrix Where n represents the total number of monitoring sites, and T represents the total number of time windows;

[0013] The PSDCN model is a lightweight model Model Based on the graph structure G and matrix X, the future spatiotemporal data can be accurately and efficiently predicted. The specific formula is as follows:

[0014]

[0015] Where: It indicates that this paper proposes the PSDCN model; represents the historical spatiotemporal data that the PSDCN model needs to input, and p represents the time-dependent step length; represents the future (predicted) spatiotemporal data, q represents the prediction step size; W represents the learnable parameters in the PSDCN model.

[0016] Furthermore, in order to better realize the present invention, the PSDCN model is composed of multiple shared parameter modules, each shared parameter module contains L spatiotemporal blocks. In the forward propagation of the model, the L spatiotemporal blocks share the same set of convolution kernels and efficiently mine the complex nonlinear dependencies in the spatiotemporal data; specifically, the parameter-sharing graph dilation convolution operator in the spatiotemporal block is used to efficiently mine the spatial dependencies in the data, and the parameter-sharing time dilation convolution operator in the spatiotemporal block is used to efficiently mine the temporal dependencies in the data.

[0017] Furthermore, in order to better implement the present invention, the forward propagation process of the PSDCN model is as follows: although the PSDCN model is composed of multiple shared parameter modules, due to the existence of the parameter-dependent module, the PSDCN model only needs one parameter-sharing module to cope with most simple spatiotemporal prediction tasks; when there is only one parameter-sharing module, the input data will pass through the convolution layer, L spatiotemporal blocks, and the convolution layer in sequence to obtain the final output; specifically, p historical spatiotemporal data First, the data dimension is increased through the convolution layer, and the input tensor of the first spatiotemporal block is obtained. Then, the tensor Model the spatiotemporal relationship of the data through L spatiotemporal blocks in sequence to obtain the output tensor of the last spatiotemporal block Finally, the tensor The data dimension is reduced through the convolution layer to obtain the final prediction result The forward propagation process of the PSDCN model is shown in the following formula

[0018]

[0019]

[0020] Where: represents the input required by the PSDCN model, and p is the history dependency step size; represents the output of the PSDCN model, q is the prediction step size; STBlock is the spatiotemporal block function in the parameter sharing module; and represents the input tensor and output tensor of the lth spatiotemporal block, and f is the data dimension of the tensor; the parameter sharing module includes L spatiotemporal blocks in total, sharing the same set of learnable parameters W T and WG ;In the spatiotemporal block, PSTDC and PSGDC represent the PSTDC operator and PSGDC operator, respectively, which are used to model the complex spatiotemporal dependencies in the data; and It is a temporary variable of the PSTDC operator and the PSGDC operator; Conv represents the convolution function, which is used for dimension alignment; Relu represents the activation function; Norm represents the parameter regularization function.

[0021] Further, in order to better implement the present invention, the input tensor in the lth space-time block is For example, the PSTDC operator calculation method is as follows:

[0022]

[0023] Where: represents the input tensor of the PSTDC operator in the lth spatiotemporal block, represents the node v in the lth space-time block i The state in the tth time window; represents the shared time convolution kernel, K represents the size of the convolution kernel, Represents the shared k-th convolution kernel; represents the dilation factor of the PSTDC operator in the lth space-time block; ⊙ represents the vector product; the convolution kernel W T The l+1th space-time block needs to be passed in for the calculation of the PSTDC operator in the l+1th space-time block.

[0024] Further, in order to better implement the present invention, after the PSTDC operator in the lth space-time block, the tensor is obtained. Tensor-based Further expand the classic graph convolution operator to efficiently mine the spatial dependencies in data under limited parameter scale, and propose a lightweight parameter-sharing graph dilation convolution operator PSGDC operator; in order to improve the prediction ability of the prediction model, the forward propagation process of the PSGDC operator is designed based on the graph attention architecture, as shown in the following formula:

[0025]

[0026]

[0027] Where: represents the input tensor of the PSGDC operator in the l-th spatiotemporal block, represents the node v in the lth space-time block i The state in the tth time window; γ jn Represents node v j For node vn The weight of influence; represents the dilation factor of the PSGDC operator in the lth spatiotemporal block; is a shared learnable parameter in the PSGDC operator; [·|·] represents the vector connection function; Relu represents the activation function; exp represents the exponential function; W S The l+1th space-time block needs to be passed in for the calculation of the PSGDC operator in the l+1th space-time block.

[0028] Furthermore, in order to better implement the present invention, S3 is, in the process of constructing the PSDCN model, multiple hyperparameters are introduced, including the history dependency step size p, the number of spatiotemporal blocks L, the dimension of the hidden layer f, the size of the temporal convolution kernel K, and the expansion factor in the PSTDC operator Dilation factor in PSGDC operator Too many hyperparameters will make it more difficult to calibrate the model. Therefore, a hyperparameter dependency component is proposed to establish the dependency relationship between hyperparameters and reduce the number of model hyperparameter calibrations.

[0029] In different space-time blocks l, when When there is a power relationship with K, the PSTDC operator can make full use of the receptive field of the shared time convolution kernel to capture a longer time window; similarly, when When there is a power relationship with 2, the PSGDC operator can model a wider spatial range; in addition, in order to make a single shared parameter module able to cope with most simple prediction tasks, it is necessary to ensure that L spatiotemporal blocks should fully model p time windows; specifically, the parameter dependency of the PSDCN model is as follows:

[0030]

[0031]

[0032]

[0033] Where: K represents the size of the shared temporal convolution kernel in the spatiotemporal block; represents the dilation factor in the PSTDC operator in the lth space-time block; represents the expansion factor in the PSGDC operator in the lth space-time block; L represents the total number of space-time blocks; after the dependency of the hyperparameters is defined, there is no need to calibrate 6 hyperparameters, only 3 hyperparameters, namely p, f and K.

[0034] Further, in order to better implement the present invention, the PSDCN model uses the spatiotemporal data {x t-p+1 , x t-p+2 ,......,xt}Predict the spatiotemporal data of q time windows in the future In the process of model optimization, the mean square error is used to optimize the predicted value. With the truth value {x t+1 , x t +2 ,......,x t+q The loss between} is as follows:

[0035]

[0036] Where: Represents node v i Observations in the t+jth time window; Represents node v i The predicted value in the t+jth time window.

[0037] Furthermore, in order to better implement the present invention, the training process of the PSDCN model is as follows: first, a training instance of the PSDCN model is constructed from the spatiotemporal matrix, then the number of spatiotemporal blocks is determined using a hyperparameter dependent component, then the expansion factor of each spatiotemporal block is calculated, and the prediction results are obtained iteratively, and finally, the proposed PSDCN model is obtained by optimizing the error loss.

[0038] The beneficial effects of the present invention are:

[0039] The PSDCN model proposed in this invention can accurately complete the spatiotemporal prediction task without significantly increasing the model calculation time / model parameter scale. In the model construction process, efficient PSTDC operators and PSGDC operators are proposed to improve the model's calculation efficiency and reduce the model's learnable weight parameter scale. In the model optimization process, the logical relationship between hyperparameters is established, the number of hyperparameter calibrations is reduced, and the model calibration efficiency is improved. In the experiment, real PM 2.5 The prediction performance of the model, including prediction accuracy, computational efficiency, parameter scale of learnable weights, and hyperparameter calibration difficulty, is verified by using the data set, traffic data set, and temperature data set. The results show that compared with the baseline methods (such as HA model, ST-KNN model, BTMF model, ST-GCN model, BiSTGN model, DSTAGNN model, STA-ODE model, GDGCN model, and STGDN model), the PSDCN model proposed in this invention has an easier hyperparameter calibration process, and exceeds or approaches the prediction accuracy of the baseline method with faster time efficiency and fewer model parameters. In addition, the influence of different components in the PSDCN model on the prediction accuracy is tested, which proves the rationality of the design of different components of the PSDCN model. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 It is the workflow diagram of the PSDCN model of the present invention;

[0041] Figure 2 Schematic diagram of forward propagation of the classical temporal convolution and parameter-sharing time-dilated convolution of the present invention; (a) classical temporal convolution, (b) parameter-sharing time-dilated convolution;

[0042] Figure 3 Schematic diagram of forward propagation of the classic graph convolution and parameter-sharing graph dilated convolution of the present invention; (a) classic graph convolution, (b) parameter-sharing graph dilated convolution;

[0043] Figure 4 Graphs illustrating the dependency of hyperparameters of the present invention; (a) hyperparameters in the PSTDC operator, (b) hyperparameters in the PSGDC operator;

[0044] Figure 5 The spatial distribution diagram of the three sets of data sets of the present invention; (a) PM 2.5 Datasets, (b) Traffic dataset, (c) Temperature dataset;

[0045] Figure 6 The stability diagram of the PSDCN model and the baseline under different random seeds of the present invention; (a) PM 2.5 Datasets, (b) Traffic dataset, (c) Temperature dataset;

[0046] Figure 7 The prediction accuracy and calculation efficiency diagram of the present invention; (a) PM 2.5 Inference time on the dataset, (b) Inference time on the traffic dataset, (c) Inference time on the temperature dataset, (d) PM 2.5 Optimization time on the dataset, (e) optimization time on the traffic dataset, (f) optimization time on the temperature dataset;

[0047] Figure 8 : is a parameter scale diagram of the prediction accuracy and learnable weight of the present invention; (a) PM 2.5 Datasets, (b) Traffic dataset, (c) Temperature dataset;

[0048] Fig. 9 The prediction result diagram of the PSDCN model of the present invention; (a) in PM 2.5 One step prediction on the dataset, (b) in PM 2.5 dataset, (c) predict one step ahead on the traffic dataset, (d) predict five steps ahead on the traffic dataset, (e) predict one step ahead on the temperature dataset, and (f) predict five steps ahead on the temperature dataset;

[0049] Fig.10It is a training process diagram of PSDCN of the present invention;

[0050] Fig.11 The prediction accuracy (RMSE / MAPE) table of the PSDCN model of the present invention and the baseline;

[0051] Fig.12 Table 2 shows the running time (milliseconds) of the PSDCN model of the present invention and the baseline, with a batch size of 256;

[0052] Fig.13 Table of parameter scales (bytes) of learnable weights for the PSDCN model of the present invention and the baseline;

[0053] Fig.14 This is a table of prediction accuracy (RMSE / MAPE) between the PSDCN model of the present invention and its components. DETAILED DESCRIPTION

[0054] The technical solutions in the embodiments of the present invention will be described clearly and completely below in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Generally, the components of the embodiments of the present invention described and shown in the accompanying drawings here can be arranged and designed in various different configurations.

[0055] Therefore, the following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the invention claimed for protection, but merely represents selected embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without making creative work are within the scope of protection of the present invention.

[0056] Figure 1-Figure 14 This is a specific embodiment of the present invention, which is an efficient parameter-sharing urban spatiotemporal data prediction method.

[0057] The spatiotemporal prediction model based on deep learning usually consists of two types of basic neural network operators, which are used to mine the temporal dependency and spatial dependency in the data respectively. The basic neural network operator can be regarded as the smallest operation unit in the spatiotemporal prediction model, which constrains the forward propagation process of the model. Common neural network operators for mining temporal dependency include recurrent neural network operators, one-dimensional convolution operators, temporal attention operators, and neural ordinary differential operators. Common neural network operators for mining spatial dependency include high-dimensional convolution operators, graph convolution operators, and spatial attention operators. Complex spatiotemporal prediction models improve the prediction accuracy of the model by improving the basic neural network operators, such as temporal graph convolution neural network, spatiotemporal graph convolution neural network with attention mechanism, dynamic spatiotemporal perception graph neural network, spatiotemporal converter, bidirectional spatiotemporal graph network, dynamic graph convolution network, spatiotemporal graph ordinary differential equation, and spatiotemporal attention ordinary differential equation. In addition, some scholars have introduced external environments such as weather and POI into the model to further improve the prediction accuracy of the spatiotemporal prediction model, such as spatiotemporal residual networks, spatiotemporal ordinary differential equations, spatiotemporal memory enhanced multi-level attention networks, and spatiotemporal event cross-attention graph convolutional neural networks. In general, with the efforts of many scholars, the existing spatiotemporal prediction models have achieved satisfactory prediction accuracy. However, with the continuous improvement of the model's prediction accuracy, the design of the model has become more and more complex, which limits the application of the model in practical scenarios. Accurate, efficient and lightweight spatiotemporal prediction models have always been the focus of academic attention, but most of the existing prediction models still find it difficult to strike a balance between model prediction accuracy and model lightweight.

[0058] Lightweight spatiotemporal prediction models aim to improve model prediction accuracy without significantly increasing model calculation time / model parameter scale. In recent years, some scholars have made some explorations on model lightweighting and established a few lightweight spatiotemporal prediction models. For example, Cheng et al. proposed a lightweight integrated spatiotemporal interpolation model to infer unknown spatiotemporal data by integrating exponential smoothing algorithm and inverse distance weight algorithm; Chien et al. proposed a lightweight cellular traffic prediction model based on convolutional neural network, and reduced the complexity of the prediction model through compression techniques such as pruning, quantization, joint training and extraction. Li et al. proposed a lightweight spatiotemporal transformer with information fusion and regional sampling. ST-TIS reduces the model calculation complexity from O(n / 2) to O(n / 2) through regional connectivity graph. 2 ) is reduced to Wang et al. proposed a lightweight spatiotemporal graph dilated convolutional network based on causal dilated convolution. ST-GDN improves the computational efficiency of spatiotemporal prediction models by reducing the depth of the neural network. In general, compared with spatiotemporal prediction models without lightweight design, the existing lightweight spatiotemporal prediction models are still very limited. In addition, the existing lightweight spatiotemporal models still have challenges such as large parameter scale and complex hyperparameter adjustment.

[0059] In view of this, this embodiment proposes a novel lightweight spatiotemporal prediction model, namely the PSDCN model. Compared with the existing lightweight spatiotemporal prediction model, the proposed PSDCN model has two advantages. First, under the premise of ensuring the prediction accuracy, the proposed PSDCN model simultaneously controls the parameter scale and computational efficiency of the model. Second, the proposed PSDCN model establishes a mathematical logical relationship between hyperparameters, which reduces the difficulty of adjusting the model parameters.

[0060] In spatiotemporal prediction applications, most spatiotemporal data can be abstracted into a graph structure. Therefore, this embodiment also constructs a PSDCN model based on a graph structure. In the graph structure G = (V, A), each monitoring site in the study area can be abstracted into a node v i ∈V, monitoring site v i With v j The relationship between them can be abstracted as edge A ij ∈A, monitoring site v i The spatiotemporal data collected within the time window t can be recorded as Generally speaking, the spatiotemporal data collected by all monitoring stations in all time windows form a matrix Where n represents the total number of monitoring sites and T represents the total number of time windows.

[0061] The research goal of this embodiment is to establish a lightweight model The proposed model M accurately and efficiently predicts future spatiotemporal data based on the graph structure G and the matrix X, as shown in formula (1).

[0062]

[0063] Where: M represents the PSDCN model proposed in this paper; represents the historical spatiotemporal data that the PSDCN model needs to input, and p represents the time-dependent step length; represents the future (predicted) spatiotemporal data, q represents the prediction step size; W represents the learnable parameters in the PSDCN model.

[0064] like Figure 1 As shown, the PSDCN model consists of multiple shared parameter modules, each of which contains L spatiotemporal blocks. In the forward propagation of the model, the L spatiotemporal blocks share the same set of convolution kernels and efficiently mine the complex nonlinear dependencies in the spatiotemporal data. Specifically, the parameter-sharing graph dilation convolution operator in the spatiotemporal block is used to efficiently mine the spatial dependencies in the data, and the parameter-sharing time dilation convolution operator in the spatiotemporal block is used to efficiently mine the temporal dependencies in the data. In addition, this embodiment studies the establishment of a hyperparameter dependency component to reduce the difficulty of hyperparameter calibration, which can optimize the model with a small number of hyperparameters.

[0065] Although the PSDCN model consists of multiple shared parameter modules, due to the existence of parameter-dependent modules, the PSDCN model only needs one parameter-sharing module to handle most simple spatiotemporal prediction tasks. When there is only one parameter-sharing module, the input data will pass through the convolutional layer, L spatiotemporal blocks, and convolutional layer in sequence to obtain the final output. Specifically, p historical spatiotemporal data First, the data dimension is increased through the convolution layer, and the input tensor of the first spatiotemporal block is obtained. Then, the tensor Model the spatiotemporal relationship of the data through L spatiotemporal blocks in sequence to obtain the output tensor of the last spatiotemporal block Finally, the tensor The data dimension is reduced through the convolution layer to obtain the final prediction result Equations (2) and (3) further define the forward propagation process of the PSDCN model using mathematical expressions.

[0066]

[0067]

[0068] Where: represents the input required by the PSDCN model, and p is the history dependency step size; represents the output of the PSDCN model, q is the prediction step size; STBlock is the spatiotemporal block function in the parameter sharing module; and represents the input tensor and output tensor of the lth spatiotemporal block, and f is the data dimension of the tensor; the parameter sharing module includes L spatiotemporal blocks in total, sharing the same set of learnable parameters W T and W G ;In the spatiotemporal block, PSTDC and PSGDC represent the PSTDC operator and PSGDC operator, respectively, which are used to model the complex spatiotemporal dependencies in the data; and is a temporary variable of the PSTDC operator and the PSGDC operator; Conv represents the convolution function, which is used for dimension alignment; Relu represents the activation function; Norm represents the parameter regularization function. From formulas (2) and (3), we can see that the core of the proposed PSDCN model lies in two points: one is the definition of the PSTDC operator in the spatiotemporal block, and the other is the definition of the PSGDC operator.

[0069] In order to solve the problems of overly complex design, low computational efficiency and large parameter scale in the model, this embodiment expands the classical time convolution operator to efficiently mine the time dependency in data under limited parameter scale. Taking a two-layer time convolutional neural network as an example, Figure 2The expansion idea of ​​the classic time convolution operator is shown. When the convolution kernel size is equal to 3, the two-layer neural network using the classic time convolution operator can only capture the dependencies of 5 time windows, while the two-layer neural network using the time expansion convolution operator can capture the dependencies of 9 time windows (the expansion factor of the first layer is ). In other words, when the historical dependency step of spatiotemporal data is fixed, the time dilation convolution operator can use fewer neural network layers to fully capture the time dependency in the data, thereby improving the computational efficiency of the model. On the other hand, this embodiment finds that the introduction of the dilation factor does not change the size of the convolution kernel in the time dilation convolution operator, but only changes the way the convolution is calculated, which makes it possible to share convolution kernels between different layers. Shared convolution kernels can ensure that the parameter scale of the model will not increase with the increase in the number of neural network layers. Inspired by the above two ideas, this embodiment proposes a lightweight parameter-sharing time dilation convolution operator (PSTDC).

[0070] Compared with the classic convolution operator, the PSTDC operator has two obvious advantages. On the one hand, the PSTDC operator controls the depth of the neural network through the expansion factor, reduces the number of forward propagation, and improves the computational efficiency of the classic time convolution operator. On the other hand, the PSTDC operator reduces the parameter scale of the classic time convolution operator by sharing the time convolution kernel between different layers. Taking as an example, formula (4) shows the calculation method of the PSTDC operator.

[0071]

[0072] Where: represents the input tensor of the PSTDC operator in the lth spatiotemporal block, represents the node v in the lth space-time block i The state in the tth time window; represents the shared time convolution kernel, K represents the size of the convolution kernel, Represents the shared k-th convolution kernel; represents the dilation factor of the PSTDC operator in the lth space-time block; ⊙ represents the vector product. Note: The convolution kernel W T The l+1th space-time block needs to be passed in for the calculation of the PSTDC operator in the l+1th space-time block.

[0073] After the PSTDC operator in the lth space-time block, this embodiment obtains the tensor Tensor-based It further expands the classic graph convolution operator to efficiently mine spatial dependencies in data under limited parameter scale. Taking a two-layer graph convolutional neural network as an example, Figure 3 The expansion idea of ​​the classic graph convolution operator is shown. For the target node, the two-layer neural network using the classic graph convolution operator can only capture the dependencies of 6 spatial neighbors, while the two-layer neural network using the dilated graph convolution operator can capture the dependencies of 9 spatial neighbors (the dilation factor of the first layer is ). This means that the introduction of the graph dilation convolution operator can capture spatial dependencies in a longer range with fewer layers, thereby improving the computational efficiency of the model. On the other hand, for most classic graph convolution operators, the introduction of the dilation factor will only change the spatial adjacency matrix of the classic graph convolution operator, and will not change the learnable parameters in the classic graph convolution operator (see Appendix A). Similarly, in the spatial dimension, the parameter scale of the model can be reduced by sharing the convolution kernel. Inspired by the above two ideas, this embodiment proposes a lightweight parameter-sharing graph dilation convolution operator.

[0074] Similar to the PSTDC operator in the time dimension, the PSGDC operator also has two obvious advantages: one is to improve the computational efficiency of the classic graph convolution operator, and the other is to reduce the parameter size of the classic graph convolution operator. In addition, in order to improve the prediction ability of the prediction model, this embodiment designs the forward propagation process of the PSGDC operator based on the graph attention architecture, as shown in formulas (5) and (6).

[0075]

[0076]

[0077] Where: represents the input tensor of the PSGDC operator in the l-th spatiotemporal block, represents the node v in the lth space-time block i The state in the tth time window; γ jn Represents node v j For node v n The weight of influence; represents the dilation factor of the PSGDC operator in the lth spatiotemporal block, calculated using the method provided by Wang et al. (2024); is a shared learnable parameter in the PSGDC operator; [·|·] represents the vector connection function; Relu represents the activation function; exp represents the exponential function. Note: W S The l+1th space-time block needs to be passed in for the calculation of the PSGDC operator in the l+1th space-time block.

[0078] In the process of constructing the PSDCN model, this embodiment introduces multiple hyperparameters, including the history dependency step size p, the number of spatiotemporal blocks L, the dimension of the hidden layer f, the size of the temporal convolution kernel K, and the expansion factor in the PSTDC operator Dilation factor in PSGDC operator As mentioned above, too many hyperparameters will increase the difficulty of model calibration. Therefore, this embodiment proposes a hyperparameter dependency component to establish a dependency relationship between hyperparameters and reduce the number of calibrations of model hyperparameters.

[0079] like Figure 4 As shown, in different space-time blocks l, when When there is a power relationship with K, the PSTDC operator can fully utilize the receptive field of the shared time convolution kernel to capture a longer time window. When there is a power relationship with 2, the PSGDC operator can model a wider spatial range. In addition, in order to make a single shared parameter module capable of handling most simple prediction tasks, it is necessary to ensure that the L spatiotemporal blocks should fully model the p time windows. Specifically, Proofs (7) to (9) define the parameter dependencies of the PSDCN model.

[0080]

[0081]

[0082] L=min l (K l-1 >p)-1 (9)

[0083] Where: K represents the size of the shared temporal convolution kernel in the spatiotemporal block; represents the dilation factor in the PSTDC operator in the lth space-time block; represents the expansion factor in the PSGDC operator in the lth space-time block; L represents the total number of space-time blocks; after the dependency of the hyperparameters is defined, there is no need to calibrate 6 hyperparameters, only 3 hyperparameters, namely p, f and K.

[0084] The PSDCN model uses the spatiotemporal data {x t-p+1 , x t-p+2 ,......,x t}Predict the spatiotemporal data of q time windows in the future In the model optimization process, this embodiment uses the mean square error to optimize the prediction value With the truth value {x t+1 , x t+2 ,......,x t+q The specific loss between} is shown in formula (10).

[0085]

[0086] Where: Represents node vi The observation value in the t+jth time window (ground truth); Represents node v i The predicted value in the t+jth time window.

[0087] The core principle of the PSDCN model is to use supervised learning methods to exploit spatiotemporal patterns in historical data to predict future trends. In order to train the proposed PSDCN model, this embodiment divides the spatiotemporal data into training samples and test samples. The training samples are used to adjust the parameters of the PSDCN model, while the test samples are used to evaluate its prediction performance. Fig.10 As shown, Algorithm 1 shows the training process of the PSDCN model. First, the training instance of the PSDCN model is constructed from the spatiotemporal matrix. Next, the number of spatiotemporal blocks is determined using the hyperparameter dependent component. Then, the dilation factor of each spatiotemporal block is calculated and the prediction results are obtained iteratively. Finally, the proposed PSDCN model is obtained by optimizing the error loss.

[0088] In this embodiment, the prediction performance of the PSDCN model is evaluated using three open source data sets, including a PM2.5 monitoring data set, a traffic flow data set, and a temperature data set.

[0089] Figure 5 The spatial distribution of three spatiotemporal datasets is shown. 2.5 The monitoring dataset contains 36 monitoring sites, and the monitoring time window is 60 minutes. The traffic flow dataset contains 71 monitoring sites, and the monitoring time window is 5 minutes. The temperature dataset contains 45 monitoring grids, and the monitoring time window is 60 minutes.

[0090] In the data preprocessing stage, we mainly completed the natural missing values ​​in the spatiotemporal data set and constructed the first-order neighbor matrix of the spatiotemporal data. Specifically, the natural missing values ​​in the spatiotemporal data are estimated by the BTTF model, and the first-order neighbors of the target spatial object are constructed by the 10 most similar spatial objects.

[0091] For spatiotemporal prediction tasks, it is very important to evaluate the prediction performance of the model, including prediction accuracy, computational efficiency, parameter scale, etc. In this embodiment, the root mean square error (RMSE) and mean absolute percentage error (MAPE) are first used to quantitatively evaluate the prediction accuracy of the PSDCN model. Secondly, the running time of forward propagation and back propagation is calculated to quantitatively evaluate the computational efficiency of the PSDCN model. Finally, the number of parameters that need to be trained is counted to quantitatively evaluate the parameter scale of the PSDCN model. Considering that the calculation method of the running time and the number of parameters is relatively simple, only the calculation method of the prediction accuracy index is listed, as shown in formulas (11)-(12).

[0092]

[0093]

[0094] Where: Represents node v i The monitoring truth in the t+j time window (ground truth); Represents the monitoring node v i Predict the value in the t+j time window; n represents the total number of graph nodes; q represents the prediction step size.

[0095] In this embodiment, 9 baseline methods are used, which can be roughly divided into two categories: ① spatiotemporal prediction model without considering lightweight design; ② spatiotemporal prediction model considering lightweight design. # Model, ST-KNN # Model, BTMF # Model, ST-GCN # Model, BiSTGN # Model, DSTAGNN # Model, STA-ODE # Model and GDGCN model. The latter is mainly STGDN & To facilitate the distinction between the types of spatiotemporal prediction models, this embodiment uses a superscript # to identify the first type of model and a superscript & to identify the second type of model.

[0096] As mentioned above, the proposed PSDCN model only needs to calibrate three hyperparameters, namely the history dependency step size p, the dimension of the hidden layer f, and the kernel size K of the PSTDC operator. In order to further reduce the difficulty of hyperparameter calibration, the history dependency step size in the three data is fixed to 6 (although a longer history dependency step size slows down the convergence of the model, it is easy to contain more information). Under the premise of fixing the history dependency step size, the control variable method is used to obtain the optimal combination of the remaining hyperparameters. 2.5 In the data set, the dimension f of the hidden layer is determined to be 40, and the kernel size K is determined to be 2. In the traffic data and temperature data sets, the dimension f of the hidden layer is determined to be 32, and the kernel size K is determined to be 2.

[0097] In order to fairly compare the prediction accuracy of the PSDCN model and the benchmark method, this example optimizes the prediction model under different random seeds and shows the optimal prediction indicators under different random seeds, such as Fig.11As shown. Overall, the prediction accuracy of the machine learning model is better than that of the statistical learning model, and the prediction accuracy of the deep learning model is better than that of the machine learning model. Specifically, on the three data sets, the prediction accuracy of the PSDCN model is much higher than that of the HA, ST-KNN, BTMF, ST-GCN, and BiSTGN models, slightly higher than that of the DSTAGNN and STA-ODE models, and close to that of the GDGCN and ST-GDN models. The results show that without considering the computational efficiency and parameter scale, the proposed PSDCN model has outperformed or achieved a prediction accuracy comparable to that of the baseline. In addition to the prediction accuracy, Figure 6 Demonstrating the stability of the prediction model with superior prediction accuracy. The results show that under different random seeds, the prediction accuracy of the proposed PSDCN model and the baseline method are relatively stable, which further proves that the proposed PSDCN model is sufficient to match the current advanced prediction models in terms of prediction accuracy.

[0098] The computational efficiency of the PSDCN model is further analyzed. Considering that the PSDCN model is a deep learning model, this embodiment only compares the PSDCN model with six deep learning models. The results are as follows: Fig.12 As shown. Among them, the inference time is mainly used to reflect the online prediction speed of the PSDCN model, and the optimization time is mainly used to reflect the offline training speed of the PSDCN model. The results show that on the three data sets, the computational efficiency of the PSDCN model is better than that of the ST-GCN model, DSTAGNN model, STA-ODE model, GDGCN model, and STGDN model, but lower than that of the BiSTGN model. This example further shows a scatter plot of computational efficiency and prediction accuracy. The results are shown in Figure 7 As shown. Figure 7 In the figure, the closer the scattered points are to the coordinate center, the better the computational efficiency and prediction accuracy of the model. The results show that although the computational efficiency of the BiSTGN model is higher, its prediction accuracy is significantly lower than that of the PSDCN model. In addition, compared with the lightweight STGDN model, the PSDCN model proposed in this embodiment also has an advantage in computational efficiency.

[0099] Similar to computational efficiency, this example analyzes the parameter scale of the learnable weights in the GSDCN model. The results are as follows: Fig.13 As shown. The results show that compared with the benchmark experiment, the GSDCN model has a significant advantage in parameter scale. We also found that compared with the lightweight STGDN model, the GSDCN model further reduces the parameter scale of the model. In addition, Figure 8A scatter plot of parameter size and prediction accuracy is shown. The results show that despite the fewer learnable parameters in the proposed GSDCN model, it still has superior prediction accuracy. The above results show that the proposed GSDCN model achieves acceptable prediction accuracy without significantly increasing the model computation time / model parameter size.

[0100] Fig. 9 Describes the difference between the predicted value and the true value. The results show that the prediction accuracy of the PSDCN model in multi-step prediction is significantly lower than that of single-step prediction, especially in the time window with large data fluctuations. Previous studies have shown that the irregular fluctuations of spatiotemporal data will affect the prediction accuracy of the model to a certain extent. Therefore, the reduction in the prediction accuracy of the PSDCN model is reasonable. In addition, although there is a significant difference in the prediction accuracy of the PSDCN model between single-step prediction and multi-step prediction, it can still accurately predict the trend of spatiotemporal data in the time dimension, which further proves that the PSDCN model has good prediction performance.

[0101] This example analyzes the prediction accuracy of the PSDCN model and its components. The results are as follows: Fig.14 As shown in the figure. The PSTDC component indicates that only the PSTDC operator is used to complete the spatiotemporal prediction task, and the PSGDC component indicates that only the PSGDC operator is used to complete the spatiotemporal prediction task. The results show that the prediction accuracy of the PSTDC component is better than that of the PSGDC component, and the prediction accuracy of the PSDCN model is better than that of the PSTDC component and the PSGDC component. The above results prove that the prediction results are more affected by temporal correlation than by spatial correlation, and it is necessary to model both temporal and spatial relationships to improve the prediction ability of the model.

[0102] The spatiotemporal prediction model is an important supporting technology for realizing real-time urban monitoring and management, and is crucial to improving urban operation efficiency and the quality of life of residents. However, most existing spatiotemporal prediction models face difficulties such as low computational efficiency, large parameter scale of learnable weights, and complex calibration of hyperparameters. In view of this, this embodiment proposes a lightweight spatiotemporal prediction model, namely the PSDCN model.

[0103] Compared with the prediction models with superior prediction accuracy, the proposed PSDCN model is not only computationally efficient, but also exceeds or approaches the prediction accuracy of multiple baseline methods. The lightweight model that takes into account both computational efficiency and prediction accuracy can be applied to spatiotemporal prediction tasks in various scenarios, especially those with high requirements for model training speed and inference speed. Compared with the existing lightweight prediction models, the proposed PSDCN model strictly controls the parameter scale of learnable weights and the number of hyperparameters, further reducing the spatial complexity and training difficulty of the model. The research results show that most spatiotemporal prediction models have obvious resource stacking phenomena and do not reasonably utilize the nonlinear fitting ability of neural networks, which also provides many prospects for the future application of the PSDCN model. On the one hand, we can introduce the ideas of parameter sharing and dilated convolution into complex models to simplify the network structure of the model without reducing the prediction accuracy. In general, we have reason to believe that the proposed PSDCN model is a very valuable spatiotemporal prediction model. On the other hand, the PSDCN model can be used as a base learner for ensemble learning, and a lightweight ensemble learning model can be established to further improve the prediction accuracy of the model.

[0104] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit it. Other modifications or equivalent substitutions made to the technical solution of the present invention by ordinary technicians in the field should be included in the scope of the claims of the present invention as long as they do not depart from the spirit and scope of the technical solution of the present invention.

Claims

1. An efficient parameter-sharing urban spatiotemporal data prediction method, characterized in that: The following steps are involved: S1, preparation stage, building the PSDCN model; S2, defining a parameter-sharing time dilation convolution operator and a parameter-sharing graph diffusion convolution operator, wherein the parameter-sharing time dilation convolution operator is a PSTDC operator, and the parameter-sharing graph diffusion convolution operator is a PSGDC operator; the parameter-sharing time dilation convolution operator and the parameter-sharing graph diffusion convolution operator can effectively and quickly capture the spatiotemporal dependency in the data without significantly increasing the model calculation time and the model parameter scale; S3, establishes the mathematical logic relationship between hyperparameters, narrows the search space of hyperparameters, reduces the difficulty of hyperparameter adjustment, reduces the time and resource investment of trial and error, and ensures the excellent performance of the PSDCN model in spatiotemporal prediction tasks; S4, three real spatiotemporal datasets are used to evaluate the performance of the PSDCN model. The three real spatiotemporal datasets are PM 2.5 data set, traffic data set, and temperature data set. The performance of the PSDCN model includes prediction accuracy, operation efficiency, and the number of model parameters.

2. The efficient parameter-sharing urban spatiotemporal data prediction method according to claim 1 is characterized in that: The PSDCN model in S1 is constructed based on a graph structure. In the graph structure G = (V, A), each monitoring site in the study area can be abstracted as a node v i ∈V, monitoring site v i With v j The relationship between them can be abstracted as edge A ij ∈A, monitoring site v i The spatiotemporal data collected within the time window t can be recorded as Generally speaking, the spatiotemporal data collected by all monitoring stations in all time windows form a matrix Where n represents the total number of monitoring sites, and T represents the total number of time windows; The PSDCN model is a lightweight model Model Based on the graph structure G and matrix X, the future spatiotemporal data can be accurately and efficiently predicted. The specific formula is as follows: Where: It indicates that this paper proposes the PSDCN model; PSDCN The model needs to input historical spatiotemporal data, and p represents the time-dependent step length; represents the future (predicted) spatiotemporal data, q represents the prediction step size; W represents the learnable parameters in the PSDCN model.

3. The efficient parameter-sharing urban spatiotemporal data prediction method according to claim 2 is characterized in that: The PSDCN model is composed of multiple shared parameter modules, each of which contains L spatiotemporal blocks. Although the PSDCN model is composed of multiple shared parameter modules, due to the existence of parameter dependency modules, the PSDCN model only needs one parameter sharing module to cope with most simple spatiotemporal prediction tasks; in the forward propagation of the model, the L spatiotemporal blocks share the same set of convolution kernels and efficiently mine the complex nonlinear dependencies in spatiotemporal data; specifically, the parameter-sharing graph dilation convolution operator in the spatiotemporal block is used to efficiently mine the spatial dependencies in the data, and the parameter-sharing time dilation convolution operator in the spatiotemporal block is used to efficiently mine the temporal dependencies in the data.

4. The efficient parameter-sharing urban spatiotemporal data prediction method according to claim 3 is characterized by: The forward propagation process of the PSDCN model is as follows: when there is only one parameter sharing module, the input data will pass through the convolution layer, L spatiotemporal blocks, and convolution layer in sequence to obtain the final output; specifically, p historical spatiotemporal data First, the data dimension is increased through the convolution layer, and the input tensor of the first spatiotemporal block is obtained. Then, the tensor Model the spatiotemporal relationship of the data through L spatiotemporal blocks in sequence to obtain the output tensor of the last spatiotemporal block Finally, the tensor The data dimension is reduced through the convolution layer to obtain the final prediction result The forward propagation process of the PSDCN model is shown in the following formula Where: represents the input required by the PSDCN model, and p is the history dependency step size; represents the output of the PSDCN model, q is the prediction step size; STBlock is the spatiotemporal block function in the parameter sharing module; and represents the input tensor and output tensor of the lth spatiotemporal block, and f is the data dimension of the tensor; the parameter sharing module includes L spatiotemporal blocks in total, sharing the same set of learnable parameters W T and W G ;In the spatiotemporal block, PSTDC and PSGDC represent the PSTDC operator and PSGDC operator, respectively, which are used to model the complex spatiotemporal dependencies in the data; and It is a temporary variable of the PSTDC operator and the PSGDC operator; Conv represents the convolution function, which is used for dimension alignment; Relu represents the activation function; Norm represents the parameter regularization function.

5. The efficient parameter-sharing urban spatiotemporal data prediction method according to claim 1 is characterized in that: Take the input tensor H in the lth spatiotemporal block l For example, the PSTDC operator calculation method is as follows: Where: represents the input tensor of the PSTDC operator in the lth spatiotemporal block, represents the node v in the lth space-time block i The state in the tth time window; represents the shared time convolution kernel, K represents the size of the convolution kernel, Represents the shared k-th convolution kernel; represents the dilation factor of the PSTDC operator in the lth space-time block; ⊙ represents the vector product; the convolution kernel W T The l+1th space-time block needs to be passed in for the calculation of the PSTDC operator in the l+1th space-time block.

6. The efficient parameter-sharing urban spatiotemporal data prediction method according to claim 5 is characterized by: After the PSTDC operator in the lth space-time block, the tensor is obtained Tensor-based Further expand the classic graph convolution operator to efficiently mine the spatial dependencies in data under limited parameter scale, and propose a lightweight parameter-sharing graph dilation convolution operator PSGDC operator; in order to improve the prediction ability of the prediction model, the forward propagation process of the PSGDC operator is designed based on the graph attention architecture, as shown in the following formula: Where: represents the input tensor of the PSGDC operator in the l-th spatiotemporal block, represents the node v in the lth space-time block i The state in the tth time window; γ jn Represents node v j For node v n The weight of influence; represents the dilation factor of the PSGDC operator in the lth spatiotemporal block; is a shared learnable parameter in the PSGDC operator; [·|·] represents the vector connection function; Relu represents the activation function; exp represents the exponential function; W S The l+1th space-time block needs to be passed in for the calculation of the PSGDC operator in the l+1th space-time block.

7. The efficient parameter-sharing urban spatiotemporal data prediction method according to claim 1 is characterized in that: The S3 is that in the process of building the PSDCN model, multiple hyperparameters are introduced, including the history dependency step size p, the number of spatiotemporal blocks L, the dimension of the hidden layer f, the size of the temporal convolution kernel K, and the expansion factor in the PSTDC operator Dilation factor in PSGDC operator Too many hyperparameters will make it more difficult to calibrate the model. Therefore, a hyperparameter dependency component is proposed to establish the dependency relationship between hyperparameters and reduce the number of model hyperparameter calibrations. In different space-time blocks l, when When there is a power relationship with K, the PSTDC operator can make full use of the receptive field of the shared time convolution kernel to capture a longer time window; similarly, when When there is a power relationship with 2, the PSGDC operator can model a wider spatial range; in addition, in order to make a single shared parameter module able to cope with most simple prediction tasks, it is necessary to ensure that L spatiotemporal blocks should fully model p time windows; specifically, the parameter dependency of the PSDCN model is as follows: Where: k represents the size of the shared temporal convolution kernel in the spatiotemporal block; represents the dilation factor in the PSTDC operator in the lth space-time block; represents the expansion factor in the PSGDC operator in the lth space-time block; L represents the total number of space-time blocks; after the dependency of the hyperparameters is defined, there is no need to calibrate 6 hyperparameters, only 3 hyperparameters, namely p, f and K.

8. The efficient parameter-sharing urban spatiotemporal data prediction method according to claim 7 is characterized by: The PSDCN model uses the spatiotemporal data of the first p time windows Predict the spatiotemporal data of q time windows in the future In the process of model optimization, the mean square error is used to optimize the predicted value. and truth value The loss between them is as follows: Where: Represents node v i Observations in the t+jth time window; Represents node v i The predicted value in the t+jth time window.

9. The efficient parameter-sharing urban spatiotemporal data prediction method according to claim 8 is characterized by: The training process of the PSDCN model is as follows: first, a training instance of the PSDCN model is constructed from the spatiotemporal matrix. Next, the number of spatiotemporal blocks is determined using a hyperparameter-dependent component. Then, the dilation factor of each spatiotemporal block is calculated and the prediction results are obtained iteratively. Finally, the proposed PSDCN model is obtained by optimizing the error loss.

Images