Model building and training methods and apparatus, graph generation methods and apparatus, media

Abstract

This disclosure relates to a model construction and training method and apparatus, a graph generation method and apparatus, and a medium. The graph generation model construction method includes: constructing a graph generation model, wherein the graph generation model includes: constructing a spatiotemporal encoder, wherein the spatiotemporal encoder is configured to generate feature vectors of traffic patterns corresponding to traffic data based on traffic data; constructing a codebook, wherein the codebook stores the correspondence between traffic patterns and latent vectors; selecting the latent vector closest to the feature vector in the codebook; and constructing a graph generator, wherein the graph generator is configured to generate feature maps of the traffic data for a spatiotemporal graph neural network based on the selected latent vector, wherein the spatiotemporal graph neural network is used for spatiotemporal prediction, and the feature map generation model includes a spatiotemporal encoder, a codebook, and a graph generator. This disclosure can capture dynamic spatial correlations with minimal modifications, improving the application performance of the model.

Description

Technical Field

[0001] This disclosure relates to the field of target detection technology, and in particular to a model building and training method and apparatus, a graph generation method and apparatus, and a medium. Background Technology

[0002] With the development of technologies such as the Internet of Things, spatiotemporal data has become the most typical data type in current urban data. Spatiotemporal prediction is a common task in urban smart application scenarios, such as urban area traffic prediction, regional sales prediction, and air quality prediction. By analyzing and mining historical spatiotemporal data, it is possible to predict the future state of urban areas and provide relevant decision support for urban managers.

[0003] The most crucial aspect of spatiotemporal prediction tasks lies in learning the spatiotemporal characteristics of spatiotemporal data. With the emergence of deep learning technology, related techniques have begun to utilize RNNs (Recurrent Neural Networks), CNNs (Convolutional Neural Networks), and GNNs (Graph Neural Networks) to capture temporal dependencies and spatial correlations in spatiotemporal data. Among these, graph neural networks are widely used to capture spatial dependencies between sensors, and some outstanding STGNNs (Spatio-Temporal Graph Neural Networks) have been proposed to solve spatiotemporal prediction problems and have achieved state-of-the-art results. Regarding the construction of spatiotemporal graphs, researchers initially used distances in the real world (such as straight-line distances or road network distances) to construct graphs and used spectral graph convolutional networks to capture spatial correlations. However, physical connections alone cannot adequately represent the spatial correlations between sensors. For example, two distant central business districts without direct physical connections may be strongly correlated because they have similar spatiotemporal patterns and can prompt each other in spatiotemporal data prediction. Therefore, some models introduce adaptive graphs to learn the potential connections between sensors. Summary of the Invention

[0004] The inventors discovered through research that related technologies overlook the dynamic spatial correlations in traffic data. For example, during the morning rush hour, people primarily move from residential areas to workplaces. Conversely, the evening rush hour often sees the opposite flow, with people returning home from the office. Furthermore, on weekends and holidays, there is a significant increase in traffic flow towards recreational destinations such as parks and shopping malls.

[0005] In view of at least one of the above technical problems, this disclosure provides a model construction and training method and apparatus, a graph generation method and apparatus, and a medium that can capture dynamic spatial correlations with minimal modifications and improve the application performance of the model.

[0006] According to one aspect of this disclosure, a method for constructing a graph generation model is provided, comprising:

[0007] Construct a spatiotemporal encoder, wherein the spatiotemporal encoder is configured to generate a feature vector of a traffic pattern corresponding to the traffic data based on traffic data;

[0008] Construct a codebook, wherein the codebook stores the correspondence between traffic patterns and latent vectors;

[0009] Select the latent vector that is closest to the feature vector from the codebook;

[0010] A graph generator is constructed, wherein the graph generator is configured to generate feature maps of the traffic data for a spatiotemporal graph neural network based on the selected latent vectors, the spatiotemporal graph neural network being used for spatiotemporal prediction, and the feature map generation model including a spatiotemporal encoder, a codebook, and a graph generator.

[0011] In some embodiments of this disclosure, the graph generation model construction method further includes:

[0012] A spatiotemporal graph neural network is constructed, wherein the spatiotemporal graph neural network performs spatiotemporal prediction based on the feature map generated by the graph generator.

[0013] In some embodiments of this disclosure, the spatiotemporal encoder includes:

[0014] A convolutional neural network layer is configured to perform convolutional processing on the traffic data in the time dimension, wherein the traffic data includes time data, sensor data, and traffic feature data, and the traffic feature data includes at least one of pedestrian flow data, vehicle flow data, and vehicle speed data;

[0015] The average pooling layer is configured to aggregate traffic feature data in the time dimension to generate information for each sensor node;

[0016] The aggregation attention layer is configured to aggregate information from each sensor node to generate a feature vector of the traffic pattern corresponding to the traffic data.

[0017] In some embodiments of this disclosure, the aggregated attention layer includes:

[0018] The attention weight determination module is configured to determine the attention weight of each sensor node based on the representation vector of each sensor node, the query matrix, and the information to be focused on.

[0019] The feature vector determination module is configured to determine the feature vector of the traffic data based on the representation vector of each sensor node and the attention weight of each sensor node.

[0020] In some embodiments of this disclosure, the graph generator includes:

[0021] A fully connected network is configured to generate point feature vectors based on the selected latent vectors;

[0022] The graph generation module is configured to generate edge feature vectors based on point feature vectors; and to generate feature graphs of the traffic data for the spatiotemporal graph neural network based on at least one of the point feature vectors and edge feature vectors.

[0023] In some embodiments of this disclosure, the graph generator further includes:

[0024] The graph generation module is also configured to concatenate multiple point feature vectors into a single point feature vector.

[0025] In some embodiments of this disclosure, selecting the latent vector closest to the feature vector in the codebook includes:

[0026] Determine the distance between the feature vector and each latent vector in the codebook;

[0027] The latent vector that has the smallest distance from the feature vector is taken as the latent vector that is closest to the feature vector.

[0028] According to another aspect of this disclosure, a model training method is provided, comprising:

[0029] A graph generation model is constructed using the model construction method described in any of the above embodiments;

[0030] The first loss function is determined based on the feature vectors of the graph generation model and the selected latent vectors.

[0031] The second loss function is determined based on the predicted and actual values ​​of the spatiotemporal graph neural network.

[0032] Based on the first loss function and the second loss function, determine the total loss function;

[0033] The graph generation model and the spatiotemporal graph neural network are trained based on the total loss function.

[0034] In some embodiments of this disclosure, determining the first loss function based on the feature vector of the graph generation model and the selected latent vector includes:

[0035] Perform constant quantization on the feature vector to determine the first distance between the selected hidden vector and the feature vector after constant quantization.

[0036] Perform constant quantization on the selected latent vector to determine the second distance between the feature vector and the constant quantized latent vector;

[0037] A first loss function is determined based on the first distance and the second distance.

[0038] According to another aspect of this disclosure, a feature map generation method is provided, comprising:

[0039] Based on the traffic data, generate a feature vector of the traffic pattern corresponding to the traffic data;

[0040] In the codebook, the latent vector that is closest to the feature vector is selected, wherein the codebook stores the correspondence between traffic patterns and latent vectors;

[0041] Based on the selected latent vectors, a feature map of the traffic data is generated for the spatiotemporal graph neural network, wherein the spatiotemporal graph neural network is used for spatiotemporal prediction.

[0042] In some embodiments of this disclosure, generating a feature vector of a traffic pattern corresponding to the traffic data includes:

[0043] The traffic data is processed by convolution in the time dimension, wherein the traffic data includes time data, sensor data, and traffic feature data;

[0044] Traffic feature data is aggregated along the time dimension to generate information for each sensor node;

[0045] The information from each sensor node is aggregated to generate a feature vector of the traffic pattern corresponding to the traffic data.

[0046] In some embodiments of this disclosure, the step of aggregating the information of each sensor node to generate a feature vector of the traffic pattern corresponding to the traffic data includes:

[0047] The attention weight of each sensor node is determined based on its representation vector, query matrix, and the information to be focused on.

[0048] Based on the representation vector of each sensor node and the attention weight of each sensor node, the feature vector of the traffic pattern corresponding to the traffic data is determined.

[0049] In some embodiments of this disclosure, generating feature maps of the traffic data for the spatiotemporal graph neural network based on selected latent vectors includes:

[0050] Generate point feature vectors based on the selected latent vectors;

[0051] Generate edge feature vectors from point feature vectors;

[0052] A feature map of the traffic data is generated for the spatiotemporal graph neural network based on at least one of the point feature vectors and edge feature vectors.

[0053] In some embodiments of this disclosure, the step of generating feature maps of the traffic data for the spatiotemporal graph neural network based on selected latent vectors further includes:

[0054] Multiple point feature vectors are concatenated into a single point feature vector.

[0055] In some embodiments of this disclosure, selecting the latent vector closest to the feature vector in the codebook includes:

[0056] Determine the distance between the feature vector and each latent vector in the codebook;

[0057] The latent vector that has the smallest distance from the feature vector is taken as the latent vector that is closest to the feature vector.

[0058] According to another aspect of this disclosure, a spatiotemporal prediction method is provided, comprising:

[0059] The feature map of the traffic data is generated using the feature map generation method described in any of the above embodiments;

[0060] Based on the feature map, a spatiotemporal graph neural network is used for spatiotemporal prediction.

[0061] According to another aspect of this disclosure, a graph generation model construction apparatus is provided, comprising:

[0062] The first construction module is configured to construct a spatiotemporal encoder, wherein the spatiotemporal encoder is configured to generate a feature vector of the traffic pattern corresponding to the traffic data based on the traffic data.

[0063] The second construction module is configured to construct a codebook, wherein the codebook stores the correspondence between traffic patterns and latent vectors;

[0064] The latent vector determination module is configured to select the latent vector that is closest to the feature vector in the codebook;

[0065] The third building module is configured to build a graph generator, wherein the graph generator is configured to generate feature maps of the traffic data for a spatiotemporal graph neural network based on the selected latent vectors, the spatiotemporal graph neural network being used for spatiotemporal prediction, and the feature map generation model including a spatiotemporal encoder, a codebook, and a graph generator.

[0066] According to another aspect of this disclosure, a model training apparatus is provided, comprising:

[0067] The graph generation model construction apparatus as described in any of the above embodiments;

[0068] The first loss determination module is configured to determine the first loss function based on the feature vectors of the model generated from the graph and the selected latent vectors.

[0069] The second loss determination module is configured to determine the second loss function based on the predicted and actual values ​​of the spatiotemporal graph neural network.

[0070] The total loss determination module is configured to determine the total loss function based on the first loss function and the second loss function;

[0071] The model training module is configured to train the graph generation model and the spatiotemporal graph neural network according to the total loss function.

[0072] According to another aspect of this disclosure, a feature map generation apparatus is provided, comprising:

[0073] A spatiotemporal encoder is configured to generate a feature vector of a traffic pattern corresponding to the traffic data, based on the traffic data.

[0074] The latent vector determination module is configured to select the latent vector that is closest to the feature vector from the codebook, wherein the codebook stores the correspondence between traffic patterns and latent vectors;

[0075] A graph generator is configured to generate feature maps of the traffic data for a spatiotemporal graph neural network based on selected latent vectors, wherein the spatiotemporal graph neural network is used for spatiotemporal prediction.

[0076] According to another aspect of this disclosure, a spatiotemporal prediction system is provided, including a spatiotemporal graph neural network and a feature map generation apparatus as described in any of the above embodiments.

[0077] According to another aspect of this disclosure, a computer apparatus is provided, comprising:

[0078] The memory is configured to store instructions;

[0079] The processor is configured to execute the instructions, causing the computer device to implement the graph generation model construction method, the model training method, the feature map generation method, or the spatiotemporal prediction method as described in any of the above embodiments.

[0080] According to another aspect of this disclosure, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores computer instructions that, when executed by a processor, implement the graph generation model construction method, the model training method, the feature map generation method, or the spatiotemporal prediction method as described in any of the above embodiments.

[0081] According to another aspect of this disclosure, a computer program product is provided, comprising a computer program, wherein the computer program, when executed by a processor, implements the graph generation model construction method, the model training method, the feature map generation method, or the spatiotemporal prediction method as described in any of the above embodiments.

[0082] This disclosure can capture dynamic spatial correlations with minimal modifications, thereby improving the application performance of the model. Attached Figure Description

[0083] To more clearly illustrate the technical solutions in the embodiments of this disclosure or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this disclosure. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0084] Figure 1 These are schematic diagrams of some embodiments of the graph generation model construction method of this disclosure.

[0085] Figure 2 Schematic diagrams of some embodiments of the model generated for this disclosure.

[0086] Figure 3 This is a schematic diagram of some embodiments of the spatiotemporal encoder disclosed herein.

[0087] Figure 4 These are schematic diagrams of some embodiments of the diagram generator of this disclosure.

[0088] Figure 5 These are schematic diagrams illustrating some embodiments of the model training method disclosed herein.

[0089] Figure 6 These are schematic diagrams illustrating some embodiments of the feature map generation method of this disclosure.

[0090] Figure 7 This is a schematic diagram of some embodiments of the spatiotemporal prediction method disclosed herein.

[0091] Figure 8 These are schematic diagrams of some embodiments of the diagram generation and model building apparatus of this disclosure.

[0092] Figure 9 Schematic diagrams of other embodiments of the diagram generation and model building apparatus of this disclosure.

[0093] Figure 10 These are schematic diagrams of some embodiments of the model training apparatus disclosed herein.

[0094] Figure 11 These are schematic diagrams of some embodiments of the feature map generation apparatus of this disclosure.

[0095] Figure 12 These are schematic diagrams of some embodiments of the spatiotemporal prediction system disclosed herein.

[0096] Figure 13 This is a schematic diagram of some embodiments of a spatiotemporal prediction system related to the technology.

[0097] Figure 14 This is a schematic diagram of some other embodiments of the spatiotemporal prediction system of related technologies.

[0098] Figure 15 This is a schematic diagram of some embodiments of the computer device disclosed herein. Detailed Implementation

[0099] The technical solutions of the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this disclosure, and not all embodiments. The following description of at least one exemplary embodiment is merely illustrative and is in no way intended to limit this disclosure or its application or use. All other embodiments obtained by those skilled in the art based on the embodiments of this disclosure without creative effort are within the scope of protection of this disclosure.

[0100] Unless otherwise specifically stated, the relative arrangement, numerical expressions, and values ​​of the components and steps set forth in these embodiments do not limit the scope of this disclosure.

[0101] At the same time, it should be understood that, for ease of description, the dimensions of the various parts shown in the accompanying drawings are not drawn according to actual scale.

[0102] Techniques, methods, and equipment known to those skilled in the art may not be discussed in detail, but where appropriate, such techniques, methods, and equipment should be considered part of the specification.

[0103] In all examples shown and discussed herein, any specific values ​​should be interpreted as merely exemplary and not as limitations. Therefore, other examples of exemplary embodiments may have different values.

[0104] It should be noted that similar labels and letters in the following figures indicate similar items; therefore, once an item is defined in one figure, it does not need to be discussed further in subsequent figures.

[0105] The inventors' research also revealed that related technologies fail to capture regular dynamic spatiotemporal patterns, resulting in poor performance in application scenarios with strong dynamic characteristics. Spatiotemporal prediction scenarios often exhibit dynamic spatiotemporal correlations; however, solutions from related technologies attempt to capture these dynamic spatiotemporal dependencies using adaptive adjacency graphs with only one set, which is significantly hindered in spatiotemporal prediction tasks such as traffic flow prediction. Furthermore, while attention-based and fully dynamic graph models possess this dynamism, they often ignore regular spatiotemporal patterns, leading to overly complex networks, low interpretability, and high computational costs.

[0106] Related technical solutions suffer from poor computational efficiency and limited practicality. Self-attention mechanisms, due to their effectiveness in capturing dependencies, have become the cornerstone of many spatiotemporal prediction models. However, their quadratic time complexity introduces significant computational overhead. Furthermore, the memory requirements of self-attention mechanisms increase quadratically with the number of nodes, limiting their application on large-scale graphs. On the other hand, fully generative graph models (such as ST-MetaNet+) offer another solution for capturing dynamic spatial dependencies, using an encoder to learn from temporal data and generate corresponding dynamic graphs. However, because the generated dynamic graphs change with different temporal inputs, the optimization of the graph convolutional module is unstable, greatly increasing the difficulty of the training process and making it very time-consuming.

[0107] Therefore, spatiotemporal graph neural networks must possess the ability to effectively identify and absorb these dynamic spatial correlations. To address this issue, this disclosure proposes a novel VQGG (Vector Quantitative Graph Generator, a lightweight dynamic adaptive graph generator module, i.e., a graph generation model). This VQGG can be considered a general-purpose plugin that can be seamlessly integrated with existing STGNNs, capturing dynamic spatial correlations with minimal modifications and improving the application performance of the model.

[0108] Therefore, the inventors have proposed a method and apparatus for constructing a graph generation model, a method and apparatus for model training, a method and apparatus for generating feature graphs, a spatiotemporal prediction method and system, a computer program product, a computer device, and a storage medium. These are described below through specific embodiments.

[0109] Figure 1 These are schematic diagrams illustrating some embodiments of the diagram generation and model construction method of this disclosure. Preferably, this embodiment can be executed by the diagram generation and model construction apparatus of this disclosure. Figure 1As shown, the graph generation model construction method of this disclosure may include steps 11 to 14.

[0110] In step 11, a spatiotemporal encoder is constructed, wherein the spatiotemporal encoder is configured to generate a feature vector of the traffic pattern corresponding to the traffic data based on the traffic data.

[0111] In step 12, a codebook is constructed, wherein the codebook stores the correspondence between traffic patterns and latent vectors.

[0112] In step 13, the latent vector that is closest to the feature vector is selected from the codebook.

[0113] In some embodiments of this disclosure, step 13 may include: determining the distance between the feature vector and each latent vector in the codebook; and taking the latent vector with the smallest distance from the feature vector as the latent vector closest to the feature vector.

[0114] In some embodiments of this disclosure, the distance may be a Euclidean distance.

[0115] In step 14, a graph generator is constructed, wherein the graph generator is configured to generate feature maps of the traffic data for a spatiotemporal graph neural network based on the selected latent vectors, the spatiotemporal graph neural network being used for spatiotemporal prediction, and the feature map generation model including a spatiotemporal encoder, a codebook, and a graph generator.

[0116] In some embodiments of this disclosure, as shown in 1, the graph generation model construction method may further include step 15.

[0117] In step 15, a spatiotemporal graph neural network is constructed, wherein the spatiotemporal graph neural network performs spatiotemporal prediction based on the feature map generated by the graph generator.

[0118] In some embodiments of this disclosure, VQGG is a general-purpose dynamic adjacency graph generator module that can be seamlessly integrated with any STGNN (Spatiotemporal Graph Neural Network).

[0119] In some embodiments of this disclosure, VQGG is used as the backbone STGNN to generate dynamic adaptive graphs.

[0120] Figure 2 These are schematic diagrams illustrating some embodiments of the model generated for this disclosure. For example... Figure 2 As shown, the graph generation model disclosed herein may include a spatiotemporal encoder (ST Encoder) 21, a codebook 22, and a graph generator 23, wherein:

[0121] The spatiotemporal encoder 21 is configured to generate a feature vector of the traffic pattern corresponding to the traffic data based on the traffic data.

[0122] In some embodiments of this disclosure, the spacetime encoder 21 may be a lightweight ST encoder.

[0123] In some embodiments of this disclosure, the spatiotemporal encoder 21 can be configured to learn the context vector of the input samples.

[0124] In some embodiments of this disclosure, the input sample may be traffic data.

[0125] In some embodiments of this disclosure, the context vector may be a feature vector of a traffic pattern.

[0126] In some embodiments of this disclosure, VQGG is merely a module for generating dynamically adaptive graphs, rather than an end-to-end network, and it does not require a complex network structure to finely learn temporal knowledge. The spatiotemporal encoder in VQGG is designed to efficiently identify the current spatiotemporal pattern of the input samples.

[0127] In some embodiments of this disclosure, VQGG employs a CNN (convolutional neural network) based spatiotemporal encoder as spatiotemporal encoder 21.

[0128] In some embodiments of this disclosure, the spatiotemporal encoder 21 can be implemented as a model such as LSTM (Long Short Term Memory), GRU (gated recurrent unit), or RNN (recurrent neural network).

[0129] Codebook 22 is configured to select the latent vector that is closest to the feature vector in the codebook, wherein the codebook pre-stores the correspondence between traffic patterns and latent vectors.

[0130] In some embodiments of this disclosure, codebook 22 can be configured to obtain the representation r through a spatiotemporal encoder. g Then, from the potential codebook Select the vector c that has the closest Euclidean distance. * The initial potential codebook is generated randomly. Here, n is the codebook size, which is a hyperparameter.

[0131] In some embodiments of this disclosure, codebook 22 may be a latent codebook.

[0132] In some embodiments of this disclosure, codebook 22 can be configured as the most recent hidden vector query.

[0133] In some embodiments of this disclosure, the latent vector may be a latent code.

[0134] In some embodiments of this disclosure, the codebook pre-stores context vectors (implicit vectors) corresponding to several major traffic modes. For example, one vector is used for the morning rush hour, another for the evening rush hour, and yet another for holidays.

[0135] Graph generator 23 is configured to generate feature maps of the traffic data for a spatiotemporal graph neural network based on the selected latent vectors, wherein the spatiotemporal graph neural network is used for spatiotemporal prediction.

[0136] In some embodiments of this disclosure, graph generator 23 may be an adjacency graph generator.

[0137] In some embodiments of this disclosure, graph generator 23 can be used to generate dynamic adjacency graphs for STGNN.

[0138] In some embodiments of this disclosure, VQGG has two main components: a lightweight ST encoder and an adjacency graph generator.

[0139] Figure 3 These are schematic diagrams illustrating some embodiments of the spatiotemporal encoder disclosed herein. Figure 3 As shown, the spatiotemporal encoder of this disclosure may include multiple convolutional neural network layers 211, average pooling layers 212, and aggregated attention layers 213.

[0140] In some embodiments of this disclosure, the spatiotemporal encoder has a sequential structure including multiple 1DCNN layers, average pooling layers, and aggregate attention layers.

[0141] The convolutional neural network layer 211 is configured to perform convolutional processing on the traffic data in the time dimension, wherein the traffic data includes time data, sensor data, and traffic feature data, and the traffic feature data includes at least one of pedestrian flow data, vehicle flow data, and vehicle speed data.

[0142] In some embodiments of this disclosure, the convolutional neural network layer 211 may be a 1D CNN layer (1-D Conv).

[0143] In some embodiments of this disclosure, 1D CNN layers can be used to efficiently learn temporal patterns.

[0144] In some embodiments of this disclosure, the input X to the 1D CNN layer can be an input sample. X is a reading of traffic data, which is a three-dimensional tensor with a size of T_in*N_s*D_in, where T_in is the number of time steps of the input, N_s is the number of sensors, and D_in is the feature dimension of the input.

[0145] In some embodiments of this disclosure, the output of the 1D CNN layer is a tensor with dimensions T_in*N_s*D_model, where D_model is the hyperparameter-model size.

[0146] The average pooling layer 212 is configured to aggregate traffic feature data in the time dimension to generate information for each sensor node.

[0147] In some embodiments of this disclosure, the average pooling layer 212 can be used to aggregate information on a time axis and generate a representation for each node.

[0148] In some embodiments of this disclosure, the average pooling layer 212 can be configured to aggregate traffic feature data (e.g., pedestrian flow data) at different times on the time axis at each node (sensor location).

[0149] In some embodiments of this disclosure, the average pooling layer 212 may be configured to employ a mean function to average the information obtained at all input time steps.

[0150] The Aggregate Attention layer (Agg.Attention) 213 is configured to aggregate information from each sensor node to generate a feature vector of the traffic pattern corresponding to the traffic data.

[0151] In some embodiments of this disclosure, the feature vector of the traffic pattern can be the context vector r. g .

[0152] In some embodiments of this disclosure, the aggregated attention layer may include an attention weight determination module and a feature vector determination module.

[0153] The attention weight determination module is configured to determine the weights based on the representation vector ri, query matrix q, and desired attention information k for each sensor node. i Determine the attention weight α for each sensor node. i .

[0154] In some embodiments of this disclosure, the attention weight determination module can be configured to determine the attention weight α of each sensor node according to formula (1). i .

[0155]

[0156] In formula (1), α i N represents the attention weights obtained after softmax. s α is the number of sensors. i This is the scaled dot product, i.e., the similarity.

[0157] In some embodiments of this disclosure, α i It can be determined according to formula (2).

[0158]

[0159] In formula (2), D model q is the hidden state dimension of the neural network layer, q is the query matrix; K is the content or primary key we want to focus on. T k i It is a dot product operation.

[0160] In some embodiments of this disclosure, q can be determined according to formula (3); k i It can be determined according to formula (4).

[0161]

[0162] k i =W K r i +b K (4)

[0163] In formulas (3) and (4), W Q , and b Q , All of them are learnable weights.

[0164] The feature vector determination module is configured to determine the feature vector based on the representation vector r of each sensor node. i And the attention weight α of each sensor node i Determine the feature vector r of the traffic data g .

[0165] In some embodiments of this disclosure, the feature vector determination module can be configured to determine the context vector r according to formula (5). g .

[0166]

[0167] In formula (5), r i It is the representation vector of the i-th sensor.

[0168] Figure 4 These are schematic diagrams illustrating some embodiments of the diagram generator of this disclosure. For example... Figure 4 As shown, the graph generator of this disclosure may include multiple fully connected networks (FC) 231 and graph generation module 232.

[0169] The fully connected network 231 is configured to generate point feature vectors based on the selected latent vectors.

[0170] In some embodiments of this disclosure, the fully connected network 231 can be configured to... * Generate point features V1 and V2.

[0171] The graph generation module 232 is configured to generate edge feature vectors based on point feature vectors; and to generate feature maps of the traffic data for the spatiotemporal graph neural network based on at least one of the point feature vectors and edge feature vectors.

[0172] In some embodiments of this disclosure, the graph generation module 232 can be configured to generate edge features E using this set of point features (point features V1, V2). ★ .

[0173] In some embodiments of this disclosure, the graph generation module may also be configured to concatenate multiple point feature vectors into a single point feature vector.

[0174] In some embodiments of this disclosure, the graph generation module 232 can be configured to determine the stitched point features V according to formula (6). ★ .

[0175]

[0176] In formula (6), || refers to the concatenation operation. Considering that some models may need to use point features to generate adaptive maps, this disclosure simply concatenates two sets of point features to form the required point features.

[0177] In some embodiments of this disclosure, the graph generation module 232 can be configured to determine the spliced ​​edge features E according to formulas (7)-(8). ★ .

[0178]

[0179] ReLU(x) = max(0,x) (8)

[0180] In formula (8), ReLU is the activation function.

[0181] In some specific embodiments of this disclosure, an adaptive adjacency matrix is ​​constructed by identifying the morning rush hour pattern, in which the edge from the residential area to the office area will have a large weight.

[0182] Figure 5 These are schematic diagrams illustrating some embodiments of the model training method of this disclosure. Preferably, this embodiment can be executed by the model training apparatus of this disclosure. Figure 5 As shown, the model training method disclosed herein may include at least one of steps 50 to 54.

[0183] Step 50: Construct a graph generation model using the model construction method described in any of the above embodiments.

[0184] Step 51: Based on the feature vectors of the graph-generated model and the selected latent vectors, determine the first loss function.

[0185] In some embodiments of this disclosure, step 51 may include: performing a constant quantization operation on the feature vector to determine a first distance between the selected latent vector and the feature vector after constant quantization; performing a constant quantization operation on the selected latent vector to determine a second distance between the feature vector and the latent vector after constant quantization; and determining a first loss function based on the first distance and the second distance.

[0186] In some embodiments of this disclosure, the addition of VQGG may result in additional losses.

[0187] In some embodiments of this disclosure, the graph generator produces a dynamic graph used in STGNNs to learn spatial correlations and predict future readouts, and obtains gradients that can be directly optimized via backpropagation. If this disclosure assumes that the context vector and the selected latent codebook are sufficiently similar, the gradients can be derived from c... * Copy to r g To bridge them. Using this method, the spatiotemporal encoder can be optimized through an end-to-end training process. In order for the assumption that the context vector and the selected latent codebook are sufficiently similar to hold, additional loss is required to reduce the distance between the context vector and the selected latent codebook, as shown in Equation (9).

[0188] In some embodiments of this disclosure, step 51 may include: determining a first loss function according to formula (9).

[0189]

[0190] In formula (9), [·] represents the constant quantization operator, which treats the parameter as a constant rather than a learnable parameter, and γ defines r. g and c ★ The ratio of their movement speeds to each other. Since a latent codebook can be used by multiple input samples, therefore c ★ The movement speed should be greater than r g Slow, i.e. γ < 1, ||||2 represents the L2 distance function, i.e., Euclidean distance.

[0191] Step 52: Determine the second loss function based on the predicted and actual values ​​of the spatiotemporal graph neural network.

[0192] In some embodiments of this disclosure, step 52 may include: based on the predicted values ​​of the spatiotemporal graph neural network. and actual value The distance function is used to determine the second loss function. Specifically, it can be done as shown in formula (10).

[0193]

[0194] In formula (10), ||||1 represents the L1 distance function, i.e., the Manhattan distance.

[0195] Step 53, according to the first loss function Second loss function Determine the total loss function.

[0196] In some embodiments of this disclosure, step 53 may include: according to formula (11), based on the first loss function Second loss function Determine the total loss function

[0197]

[0198] In formula (11), τ is a hyperparameter.

[0199] Step 54: Train the graph generation model and the spatiotemporal graph neural network according to the total loss function.

[0200] In some embodiments of this disclosure, step 54 may include: optimizing the model parameters of the graph generation model and the spatiotemporal graph neural network according to the total loss function, wherein the model parameters may include: the weights of the neural network modules, such as the matrix W and vector b for a fully connected network; the values ​​of the convolution kernels of the convolutional network, etc.

[0201] Figure 6 These are schematic diagrams illustrating some embodiments of the feature map generation method of this disclosure. Preferably, this embodiment can be executed by the feature map generation apparatus of this disclosure. Figure 6 As shown, the feature map generation method of this disclosure may include at least one of steps 61 to 63.

[0202] Step 61: Generate a feature vector of the traffic pattern corresponding to the traffic data.

[0203] In some embodiments of this disclosure, step 61 may include at least one of steps 611 to 613.

[0204] Step 611: Perform convolution processing on the traffic data in the time dimension, wherein the traffic data includes time data, sensor data, and traffic feature data.

[0205] Step 612: Aggregate traffic feature data in the time dimension to generate information for each sensor node.

[0206] Step 613: Collect the information from each node and generate a feature vector of the traffic mode corresponding to the traffic data.

[0207] In some embodiments of this disclosure, step 613 may include: aggregating the information of each sensor node to generate a feature vector of the traffic pattern corresponding to the traffic data.

[0208] In some embodiments of this disclosure, step 613 may include: determining the attention weight of each sensor node based on the representation vector of each sensor node, the query matrix, and the information to be focused on; and determining the feature vector of the traffic pattern corresponding to the traffic data based on the representation vector of each sensor node and the attention weight of each sensor node.

[0209] Step 62: Select the latent vector closest to the feature vector from the codebook, wherein the codebook stores the correspondence between traffic patterns and latent vectors. In some embodiments of this disclosure, step 62 may include: determining the distance between the feature vector and each latent vector in the codebook; and selecting the latent vector with the smallest distance to the feature vector as the latent vector closest to the feature vector.

[0210] Step 63: Based on the selected latent vector, generate a feature map of the traffic data for the spatiotemporal graph neural network, wherein the spatiotemporal graph neural network is used for spatiotemporal prediction.

[0211] In some embodiments of this disclosure, step 63 may include at least one of steps 631 to 633.

[0212] Step 631: Generate point feature vectors based on the selected latent vectors.

[0213] Step 632: Generate edge feature vectors based on point feature vectors;

[0214] Step 633: Generate a feature map of the traffic data for the spatiotemporal graph neural network based on at least one of the point feature vectors and edge feature vectors.

[0215] In some embodiments of this disclosure, step 63 may further include: concatenating multiple point feature vectors into a single point feature vector.

[0216] In some embodiments of this disclosure, the feature map generation method is executed by a graph generation model, which is integrated into the spatiotemporal graph neural network.

[0217] Figure 7 These are schematic diagrams illustrating some embodiments of the spatiotemporal prediction method of this disclosure. Preferably, this embodiment can be executed by the spatiotemporal prediction system of this disclosure. Figure 7 As shown, the feature map generation method of this disclosure may include at least one of steps 71 to 72.

[0218] Step 71: Generate a feature map of the traffic data using the feature map generation method described in any of the above embodiments.

[0219] Step 72: Based on the feature map, a spatiotemporal graph neural network is used to perform spatiotemporal prediction.

[0220] The spatiotemporal prediction method and feature map generation method disclosed herein possess the ability to effectively identify and absorb these dynamic spatial correlations. In the embodiments of this disclosure, the same dynamic feature map is used for the morning peak and a different dynamic feature map is used for the evening peak.

[0221] This disclosure proposes a novel lightweight dynamic adaptive graph generator module (graph generation model) that can be seamlessly integrated with existing STGNNs in related technologies to capture dynamic spatial correlations with minimal modifications and improve the application performance of the model.

[0222] This disclosure can capture regular dynamic spatiotemporal patterns and performs well in application scenarios with strong dynamic characteristics.

[0223] The embodiments of this disclosure enable dynamic spatiotemporal correlation learning. To efficiently capture dynamic spatial correlations, the embodiments of this disclosure design an efficient spatiotemporal encoder, a latent codebook, and a two-step graph adaptive generator.

[0224] The embodiments of this disclosure reduce computational overhead, model complexity, and improve efficiency. The spatiotemporal encoders of the embodiments of this disclosure are based on 1D (one-dimensional) CNNs to better utilize hardware accelerators such as GPUs (Graphics Processing Units) and NPUs (Neural Network Processing Units) to reduce computational overhead. The two-step graph generators of the embodiments of this disclosure provide better compatibility with STGNNs than related single-step generators and have fewer parameters.

[0225] Figure 8 Schematic diagrams of some embodiments of the model building apparatus for generating diagrams in this disclosure. For example... Figure 8As shown, the diagram generation model building apparatus of this disclosure may include a first building module 81, a second building module 82, an implicit vector determination module 83, and a third building module 84.

[0226] The first construction module 81 is configured to construct a spatiotemporal encoder, wherein the spatiotemporal encoder is configured to generate a feature vector of the traffic pattern corresponding to the traffic data based on the traffic data.

[0227] In some embodiments of this disclosure, the graph generation model can be as follows: Figure 2 The graph generation model shown in the embodiment.

[0228] In some embodiments of this disclosure, the spatiotemporal encoder can be as follows: Figure 3 The spatiotemporal encoder of the embodiment.

[0229] The second construction module 82 is configured to construct a codebook, wherein the codebook stores the correspondence between traffic patterns and latent vectors.

[0230] The latent vector determination module 83 is configured to select the latent vector that is closest to the feature vector in the codebook.

[0231] The third construction module 84 is configured to construct a graph generator, wherein the graph generator is configured to generate feature maps of the traffic data for a spatiotemporal graph neural network based on the selected latent vectors, the spatiotemporal graph neural network being used for spatiotemporal prediction, and the feature map generation model including a spatiotemporal encoder, a codebook, and a graph generator.

[0232] In some embodiments of this disclosure, the graph generator can be as follows: Figure 4 The graph generator in the example.

[0233] Figure 9 Schematic diagrams of other embodiments of the model building apparatus for generating diagrams in this disclosure. Figure 8 Compared to the previous examples, Figure 9 The graph generation model building apparatus of the embodiment may further include a fourth building module 85.

[0234] The fourth construction module 85 is configured to construct a spatiotemporal graph neural network, wherein the spatiotemporal graph neural network performs spatiotemporal prediction based on the feature map generated by the graph generator; the graph generation model is integrated into the spatiotemporal graph neural network.

[0235] In some embodiments of this disclosure, the graph generation model building apparatus of this disclosure can be used to perform the graph generation model building method as described in any of the above embodiments.

[0236] Figure 10 These are schematic diagrams illustrating some embodiments of the model training apparatus disclosed herein. Figure 11As shown, the model training apparatus disclosed herein may include a graph generation model building apparatus 100, a first loss determination module 101, a second loss determination module 102, a total loss determination module 103, and a model training module 104.

[0237] The graph generation model building apparatus 100 can be any of the embodiments described above (e.g., Figure 8 or Figure 9 The graph generation model construction apparatus described in the embodiment)

[0238] The first loss determination module 101 is configured to determine the first loss function based on the feature vector of the model generated from the graph and the selected latent vector.

[0239] In some embodiments of this disclosure, the first loss determination module 101 may be configured to perform a constant quantization operation on the feature vector to determine a first distance between the selected latent vector and the feature vector after the constant quantization operation; perform a constant quantization operation on the selected latent vector to determine a second distance between the feature vector and the latent vector after the constant quantization operation; and determine a first loss function based on the first distance and the second distance.

[0240] The second loss determination module 102 is configured to determine a second loss function based on the predicted and actual values ​​of the spatiotemporal graph neural network.

[0241] The total loss determination module 103 is configured to determine the total loss function based on the first loss function and the second loss function.

[0242] The model training module 104 is configured to train the graph generation model and the spatiotemporal graph neural network according to the total loss function.

[0243] In some embodiments of this disclosure, the model training apparatus of this disclosure can be used to perform the model training method as described in any of the above embodiments.

[0244] Figure 11 These are schematic diagrams illustrating some embodiments of the feature map generation apparatus of this disclosure. For example... Figure 11 As shown, the feature map generation apparatus of this disclosure may include a spatiotemporal encoder 111, a latent vector determination module 112, and a graph generator 113.

[0245] The spatiotemporal encoder 111 is configured to generate a feature vector of the traffic pattern corresponding to the traffic data based on the traffic data.

[0246] In some embodiments of this disclosure, the spatiotemporal encoder 111 can be configured to perform convolution processing on the traffic data in the time dimension, wherein the traffic data includes time data, sensor data, and traffic feature data; aggregate the traffic feature data in the time dimension to generate information for each sensor node; and aggregate the information for each sensor node to generate a feature vector of the traffic pattern corresponding to the traffic data.

[0247] In some embodiments of this disclosure, the spatiotemporal encoder 111 can be configured to, when aggregating the information of each sensor node to generate a feature vector of the traffic pattern corresponding to the traffic data, determine the attention weight of each sensor node based on the representation vector of each sensor node, the query matrix, and the information to be focused on; and determine the feature vector of the traffic pattern corresponding to the traffic data based on the representation vector of each sensor node and the attention weight of each sensor node.

[0248] The latent vector determination module 112 is configured to select the latent vector that is closest to the feature vector from the codebook, wherein the codebook stores the correspondence between traffic patterns and latent vectors.

[0249] In some embodiments of this disclosure, the latent vector determination module 112 can be configured to determine the distance between the feature vector and each latent vector in the codebook; and to take the latent vector with the smallest distance from the feature vector as the latent vector closest to the feature vector.

[0250] Graph generator 113 is configured to generate feature maps of the traffic data for a spatiotemporal graph neural network based on selected latent vectors, wherein the spatiotemporal graph neural network is used for spatiotemporal prediction.

[0251] In some embodiments of this disclosure, the graph generator 113 may be configured to generate point feature vectors based on the selected latent vectors; generate edge feature vectors based on the point feature vectors; and generate feature graphs of the traffic data for the spatiotemporal graph neural network based on at least one of the point feature vectors and edge feature vectors.

[0252] In some embodiments of this disclosure, the graph generator 113 may also be configured to concatenate multiple point feature vectors into a single point feature vector.

[0253] In some embodiments of this disclosure, the feature map generation method may be executed by a graph generation model integrated into the spatiotemporal graph neural network.

[0254] In some embodiments of this disclosure, the feature map generation apparatus of this disclosure can be used to perform the feature map generation method as described in any of the above embodiments.

[0255] Figure 12These are schematic diagrams illustrating some embodiments of the spatiotemporal prediction system disclosed herein. Figure 12 As shown, the spatiotemporal prediction system disclosed herein may include a spatiotemporal graph neural network 121 and a feature map generation device 122.

[0256] In some embodiments of this disclosure, the feature map generation apparatus 122 may be a feature map generation apparatus as described in any of the above embodiments.

[0257] In some embodiments of this disclosure, the feature map generation apparatus 122 may be a graph generation model as described in any of the above embodiments.

[0258] In some embodiments of this disclosure, the spatiotemporal prediction system of this disclosure can be used to perform the spatiotemporal prediction method as described in any of the above embodiments.

[0259] In some embodiments of this disclosure, the feature map generation device 122 may be VQGG.

[0260] In some embodiments of this disclosure, the spatiotemporal graph neural network 121 may be an STGNN.

[0261] In some embodiments of this disclosure, the feature map generation apparatus 122 can be configured to generate an adaptive feature map (including point features V) based on an input traffic data sample X. ★ Edge feature E ★ ).

[0262] In some embodiments of this disclosure, the spatiotemporal graph neural network 121 can be configured to predict traffic data based on an adaptive feature map, input traffic data samples X, sensor node tensors V, and tensors ε of edges between sensors.

[0263] Figure 13 This is a schematic diagram of some embodiments of a spatiotemporal prediction system related to the technology. Figure 14 This is a schematic diagram of some other embodiments of the spatiotemporal prediction system of related technologies. Figure 13 The spatiotemporal prediction system in this embodiment only includes a spatiotemporal graph neural network (STGNN). Figure 13 There is no adaptive feature map. Figure 14 The spatiotemporal prediction system in this embodiment is an STGNN with an adaptive graph. This disclosure Figure 12 The spatiotemporal prediction system in this embodiment is an STGNN with VQGG appended (VQ-STGN).

[0264] Figure 12In this embodiment, the spatiotemporal graph neural network 121 is the baseline model, and the subsequent models can be the Adaptive Graph Convolutional Recurrent Network (AGCRN) or the Graph WaveNet (GWNet) model.

[0265] The method disclosed herein has been validated through comparative experiments on two commonly used datasets, PEMS04 and PEMS08. Detailed experimental results are shown in Table 1. In Table 1, the baseline models selected are the Adaptive Graph Convolutional Recurrent Network (AGCRN) and the Graph WaveNet (GWNet). To validate the method disclosed herein, the VQGG method was integrated into the baseline models, denoted as VQ-AGCRN and VQ-GWNet respectively, and the model performance before and after integration was compared experimentally.

[0266]

[0267] Table 1

[0268] Table 1 shows the performance comparison of VQGG. In Table 1, RMSE is the Root Mean Square Error; metrics are the indicators, and Rel imp is the relative improvement. According to the results shown in Table 1, the VQ-STGNs (VQ-AGCRN and VQ-GWNet) based on the method of this disclosure consistently outperform the baseline model (the performance of the baseline model is from https: / / github.com / zezhishao / BasicTS). This disclosure conducted multiple experiments on VQ-STGN using five different random seeds, and the results are expressed as "mean ± std". VQ-STGNs (VQ-AGCRN and VQ-GWNet) outperform the baseline model on all metrics. On the PEMS04 dataset, integrating VQGG into AGCRN can reduce MAE (Mean Absolute Error) by 2.59%. Furthermore, VQGG is observed to be particularly effective in improving long-term prediction accuracy. Specifically, when VQGG is added to Graph WaveNet, the 60-minute MAE shows a greater improvement margin on both datasets compared to the 15-minute MAE. This phenomenon was also observed in VQ-AGCRN on the PEMS08 dataset. This is reasonable because VQGG provides an adaptive graph for each input sample based on traffic patterns. By using an appropriate adaptive graph, the model can better capture spatial correlations in the data. In the context of traffic prediction, the accuracy of long-term predictions depends on the model's ability to effectively capture these spatial patterns.

[0269]

[0270] Table 2

[0271] Table 2 compares the runtime of this disclosure. In Table 2, w / o means without; w / means with; OOM means memory exhaustion; N / A means not present; and Rel Diff is the relative difference. Table 2 shows the execution time required for each round of training of the baseline model and VQ-STGN on the same NVIDIA RTX4090. The results in Table 2 show that VQ-STGN needs to compute the adaptive map separately for each sample, thus introducing additional computation. This additional computation is linearly related to the batch size. If the additional computation is amortized across each sample, such as the additional computation cost of VQ-GWNet which is about 1%, the additional computation introduced by the method of this disclosure is acceptable considering the improvement in accuracy.

[0272] For spatiotemporal prediction tasks, and addressing technical challenges such as dynamic and complex spatiotemporal patterns and computational efficiency encountered in real-world scenarios, the above embodiments of this disclosure propose a novel lightweight dynamic adaptive graph generator module. The above embodiments of this disclosure can be regarded as a general-purpose plugin that can be seamlessly integrated with existing STGNNs to capture dynamic spatial correlations with minimal modifications, thereby improving the application performance of the model.

[0273] To reduce the number of model parameters, the above embodiments of this disclosure design a two-step graph generator method, which can provide better compatibility with STGNNs than a single-step generator and has fewer parameters.

[0274] Figure 15 This is a schematic diagram illustrating the structure of some embodiments of the computer device disclosed herein. For example... Figure 15 As shown, the computer device includes a memory 401 and a processor 402.

[0275] The memory 401 is used to store instructions, and the processor 402 is coupled to the memory 401. The processor 402 is configured to execute the graph generation model construction method, the model training method, the feature map generation method, or the spatiotemporal prediction method as described in any of the above embodiments, based on the instructions stored in the memory.

[0276] like Figure 15 As shown, the computer device also includes a communication interface 403 for exchanging information with other devices. Additionally, the computer device includes a bus 404, through which the processor 402, communication interface 403, and memory 401 communicate with each other.

[0277] The memory 401 may include high-speed RAM, and may also include non-volatile memory, such as at least one disk drive. The memory 401 may also be a memory array. The memory 401 may also be divided into blocks, and these blocks may be combined into virtual volumes according to certain rules.

[0278] Furthermore, processor 402 may be a central processing unit (CPU), an application-specific integrated circuit (ASIC), or one or more integrated circuits configured to implement embodiments of the present disclosure.

[0279] According to another aspect of this disclosure, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores computer instructions that, when executed by a processor, implement the graph generation model construction method, the model training method, the feature map generation method, or the spatiotemporal prediction method as described in any of the above embodiments.

[0280] In some embodiments of this disclosure, the computer-readable storage medium may be a non-transitory computer-readable storage medium.

[0281] According to another aspect of this disclosure, a computer program product is provided, comprising a computer program, wherein the computer program, when executed by a processor, implements the graph generation model construction method, the model training method, the feature map generation method, or the spatiotemporal prediction method as described in any of the above embodiments.

[0282] The methods provided in this disclosure can also be configured as a program product including a readable medium storing the program code. This readable medium can take many forms, such as a readable signal medium or a readable storage medium, such as an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device. Specifically, the readable storage medium can include, but is not limited to, electrical connections, portable disks, hard disks, RAM, ROM, erasable programmable read-only memory (EPROM), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, etc.

[0283] The graph generation model building apparatus, first building module, second building module, third building module, fourth building module, latent vector determination module, model training apparatus, first loss determination module, second loss determination module, total loss determination module, model training module, spatiotemporal encoder, graph generator, codebook, spatiotemporal graph neural network, feature graph generation apparatus, and computer apparatus described above can be implemented as a general-purpose processor, programmable logic controller (PLC), digital signal processor (DSP), application-specific integrated circuit (ASIC), field-programmable gate array (FPGA), or other programmable logic device, discrete gate or transistor logic device, discrete hardware component, or any suitable combination thereof for performing the functions described in this application.

[0284] This concludes the detailed description of the present disclosure. To avoid obscuring the concept of the disclosure, some details known in the art have not been described. Those skilled in the art will fully understand how to implement the technical solutions disclosed herein based on the above description.

[0285] Those skilled in the art will understand that all or part of the steps of the above embodiments can be implemented by hardware, or by a program instructing the relevant hardware to implement them. The program can be stored in a computer-readable storage medium, such as a read-only memory, a disk, or an optical disk.

[0286] The description in this disclosure is provided for illustrative and descriptive purposes only and is not intended to be exhaustive or to limit the disclosure to its forms. Many modifications and variations will be apparent to those skilled in the art. The embodiments were chosen and described in order to better illustrate the principles and practical application of this disclosure and to enable those skilled in the art to understand this disclosure and to design various embodiments with various modifications suitable for a particular purpose.

Claims

1. A method for constructing a graph generation model, comprising: Construct a spatiotemporal encoder, wherein the spatiotemporal encoder is configured to generate a feature vector of a traffic pattern corresponding to the traffic data based on traffic data; Construct a codebook, wherein the codebook stores the correspondence between traffic patterns and latent vectors; Select the latent vector that is closest to the feature vector from the codebook; A graph generator is constructed, wherein the graph generator is configured to generate feature maps of the traffic data for a spatiotemporal graph neural network based on the selected latent vectors, the spatiotemporal graph neural network being used for spatiotemporal prediction, and the feature map generation model includes a spatiotemporal encoder, a codebook, and a graph generator.

2. The graph generation model construction method according to claim 1 further includes: A spatiotemporal graph neural network is constructed, wherein the spatiotemporal graph neural network performs spatiotemporal prediction based on the feature map generated by the graph generator.

3. The graph generation model construction method according to claim 1 or 2, wherein, The spatiotemporal encoder includes: A convolutional neural network layer is configured to perform convolutional processing on the traffic data in the time dimension, wherein the traffic data includes time data, sensor data, and traffic feature data, and the traffic feature data includes at least one of pedestrian flow data, vehicle flow data, and vehicle speed data; The average pooling layer is configured to aggregate traffic feature data in the time dimension to generate information for each sensor node; The aggregation attention layer is configured to aggregate information from each sensor node to generate a feature vector of the traffic pattern corresponding to the traffic data.

4. The graph generation model construction method according to claim 3, wherein, The aggregated attention layer includes: The attention weight determination module is configured to determine the attention weight of each sensor node based on the representation vector of each sensor node, the query matrix, and the information to be focused on. The feature vector determination module is configured to determine the feature vector of the traffic data based on the representation vector of each sensor node and the attention weight of each sensor node.

5. The graph generation model construction method according to claim 1 or 2, wherein, The graph generator includes: A fully connected network is configured to generate point feature vectors based on the selected latent vectors; The graph generation module is configured to generate edge feature vectors based on point feature vectors; and to generate feature graphs of the traffic data for the spatiotemporal graph neural network based on at least one of the point feature vectors and edge feature vectors.

6. The graph generation model construction method according to claim 5, wherein: The graph generation module is also configured to concatenate multiple point feature vectors into a single point feature vector.

7. The graph generation model construction method according to claim 1 or 2, wherein, The step of selecting the latent vector that is closest to the feature vector in the codebook includes: Determine the distance between the feature vector and each latent vector in the codebook; The latent vector that has the smallest distance from the feature vector is taken as the latent vector that is closest to the feature vector.

8. A model training method, comprising: A graph generation model is constructed using the model construction method as described in any one of claims 1 to 7; The first loss function is determined based on the feature vectors of the graph generation model and the selected latent vectors. The second loss function is determined based on the predicted and actual values ​​of the spatiotemporal graph neural network. Based on the first loss function and the second loss function, determine the total loss function; The graph generation model and the spatiotemporal graph neural network are trained based on the total loss function.

9. The model training method according to claim 8, wherein, The step of determining the first loss function based on the feature vector of the graph-generated model and the selected latent vector includes: Perform constant quantization on the feature vector to determine the first distance between the selected hidden vector and the feature vector after constant quantization. Perform constant quantization on the selected latent vector to determine the second distance between the feature vector and the constant quantized latent vector; A first loss function is determined based on the first distance and the second distance.

10. A feature map generation method, comprising: Based on the traffic data, generate a feature vector of the traffic pattern corresponding to the traffic data; In the codebook, the latent vector that is closest to the feature vector is selected, wherein the codebook stores the correspondence between traffic patterns and latent vectors; Based on the selected latent vectors, a feature map of the traffic data is generated for the spatiotemporal graph neural network, wherein the spatiotemporal graph neural network is used for spatiotemporal prediction.

11. The feature map generation method according to claim 10, wherein, The step of generating a feature vector of the traffic pattern corresponding to the traffic data includes: The traffic data is processed by convolution in the time dimension, wherein the traffic data includes time data, sensor data, and traffic feature data; Traffic feature data is aggregated along the time dimension to generate information for each sensor node; The information from each sensor node is aggregated to generate a feature vector of the traffic pattern corresponding to the traffic data.

12. The feature map generation method according to claim 11, wherein, The process of aggregating information from each sensor node to generate a feature vector for the traffic pattern corresponding to the traffic data includes: The attention weight of each sensor node is determined based on its representation vector, query matrix, and the information that needs to be focused on. Based on the representation vector of each sensor node and the attention weight of each sensor node, the feature vector of the traffic pattern corresponding to the traffic data is determined.

13. The feature map generation method according to any one of claims 10 to 12, wherein, The process of generating feature maps of the traffic data using the selected latent vectors for the spatiotemporal graph neural network includes: Generate point feature vectors based on the selected latent vectors; Generate edge feature vectors from point feature vectors; A feature map of the traffic data is generated for the spatiotemporal graph neural network based on at least one of the point feature vectors and edge feature vectors.

14. The feature map generation method according to claim 13, wherein, The process of generating feature maps of the traffic data using the selected latent vectors for the spatiotemporal graph neural network further includes: Multiple point feature vectors are concatenated into a single point feature vector.

15. The feature map generation method according to any one of claims 10 to 12, wherein, The step of selecting the latent vector that is closest to the feature vector in the codebook includes: Determine the distance between the feature vector and each latent vector in the codebook; The latent vector that has the smallest distance from the feature vector is taken as the latent vector that is closest to the feature vector.

16. A spatiotemporal prediction method, comprising: The feature map of the traffic data is generated using the feature map generation method as described in any one of claims 10-15; Based on the feature map, a spatiotemporal graph neural network is used for spatiotemporal prediction.

17. A graph generation model construction apparatus, comprising: The first construction module is configured to construct a spatiotemporal encoder, wherein the spatiotemporal encoder is configured to generate a feature vector of the traffic pattern corresponding to the traffic data based on the traffic data. The second construction module is configured to construct a codebook, wherein the codebook stores the correspondence between traffic patterns and latent vectors; The latent vector determination module is configured to select the latent vector that is closest to the feature vector in the codebook; The third building module is configured to build a graph generator, wherein the graph generator is configured to generate feature maps of the traffic data for a spatiotemporal graph neural network based on the selected latent vectors. The spatiotemporal graph neural network is used for spatiotemporal prediction, and the feature map generation model includes a spatiotemporal encoder, a codebook, and a graph generator.

18. A model training device, comprising: The graph generation model construction apparatus as described in claim 17; The first loss determination module is configured to determine the first loss function based on the feature vectors of the model generated from the graph and the selected latent vectors. The second loss determination module is configured to determine the second loss function based on the predicted and actual values ​​of the spatiotemporal graph neural network. The total loss determination module is configured to determine the total loss function based on the first loss function and the second loss function; The model training module is configured to train the graph generation model and the spatiotemporal graph neural network according to the total loss function.

19. A feature map generation apparatus, comprising: A spatiotemporal encoder is configured to generate a feature vector of a traffic pattern corresponding to the traffic data, based on the traffic data. The latent vector determination module is configured to select the latent vector that is closest to the feature vector from the codebook, wherein the codebook stores the correspondence between traffic patterns and latent vectors; A graph generator is configured to generate feature maps of the traffic data for a spatiotemporal graph neural network based on selected latent vectors, wherein the spatiotemporal graph neural network is used for spatiotemporal prediction.

20. A spatiotemporal prediction system, comprising a spatiotemporal graph neural network and a feature map generation device as described in claim 19.

21. A computer device comprising: The memory is configured to store instructions; A processor is configured to execute the instructions such that the computer device implements the graph generation model construction method as described in any one of claims 1-7, the model training method as described in claim 8 or 9, the feature map generation method as described in any one of claims 10-15, or the spatiotemporal prediction method as described in claim 16.

22. A computer-readable storage medium, wherein, The computer-readable storage medium stores computer instructions that, when executed by a processor, implement the graph generation model construction method as described in any one of claims 1-7, the model training method as described in claim 8 or 9, the feature map generation method as described in any one of claims 10-15, or the spatiotemporal prediction method as described in claim 16.

23. A computer program product comprising a computer program, wherein, When the computer program is executed by the processor, it implements the graph generation model construction method as described in any one of claims 1-7, the model training method as described in claim 8 or 9, the feature map generation method as described in any one of claims 10-15, or the spatiotemporal prediction method as described in claim 16.

Images