Data completion method and system based on space-time attention, electronic equipment and medium

Abstract

The invention discloses a data completion method and system based on space-time attention, electronic equipment and a medium, and the method comprises the steps: constructing a multi-head self-attention network through self-attention, extracting potential spatial features in traffic data, constructing a graph attention feedforward network through a graph attention layer and a feedforward neural network, and completing the data completion through the graph attention feedforward network. Real spatial features in the traffic data are extracted, and comprehensive extraction of the spatial features is realized by fusing the two features; a time attention module is used for extracting time correlation, and the spatial features and the time features of the traffic data are extracted in sequence and fused, so that comprehensive extraction of the spatial-temporal features is realized. According to the method, a traffic data completion framework based on space-time attention is constructed, completion operation is executed in a distributed manner by adopting a completion optimization serial mode, a completion result is optimized by using an iteration strategy, and the method can be suitable for data completion tasks in various data missing scenes.

Description

Technical Field

[0001] The present invention relates to the field of data security technology, and in particular to a data completion method, system, electronic device and medium based on spatiotemporal attention. Background Art

[0002] Intelligent Transportation System (ITS) refers to the construction of a comprehensive network interconnection for the transportation system with the help of advanced information and communication technology, sensors, intelligent devices, etc., so as to achieve data collection, analysis and intelligent control in the traffic environment. It is the key foundation for the transformation of modern urban transportation, logistics and transportation and other fields towards efficient and intelligent transportation modes. The application of technologies such as Internet of Vehicles, cloud computing offloading, deep mining of big data and high-precision sensing has greatly promoted the progress of intelligent transportation systems. Its performance has become more and more excellent, and the complexity of the system has gradually increased. Relying on various types of traffic sensor equipment and large-scale transmission of traffic network data, the intelligent transportation system can achieve vehicle interconnection, human-vehicle interaction, road condition transmission, environmental detection and intelligent judgment, and based on the all-round deep perception of traffic operation information, massive data flow, real-time and fast calculation and precise modeling and analysis, it can realize intelligent traffic command, travel strategy optimization and management system innovation. However, in actual scenarios, due to factors such as extreme weather, equipment failure, and communication interruption, the data collected by the transportation network is often missing, which seriously restricts the development of smart transportation. Therefore, analyzing the spatiotemporal characteristics of traffic data and thus completing the missing traffic data is of great significance for maintaining the data security of intelligent transportation systems.

[0003] In recent years, a lot of research has been carried out at home and abroad on traffic data completion. Existing traffic data completion methods can be roughly divided into five categories: 1) prediction-based methods; 2) interpolation-based methods; 3) statistical learning-based methods; 4) matrix / tensor completion methods; 5) deep learning methods.

[0004] Among them, deep learning methods are the focus of current research. This type of method extracts spatial and temporal correlations in traffic data based on a variety of neural networks to fill in missing values. Through special design, deep learning methods can solve the defects of the above four methods and can cope with different missing patterns and different missing rates while maintaining a good effect. Among them, the method based on denoising autoencoder reconstructs the data for filling by eliminating noise, such as denoising stacked autoencoder (DSAE) and variational autoencoder (VAE). Some methods use recurrent neural network (RNN) to extract the temporal correlation of sequence data to achieve data filling and achieve excellent performance. Given that generative adversarial network (GAN) is very popular in the field of data generation, many data filling methods use it to generate missing data, such as generative adversarial filling network (GAIN) and GA-GAN, which learn the distribution law of real data through adversarial games between generators and discriminators. In addition, the breakthrough of graph convolutional network (GCN) has brought new ideas for traffic data repair. Because graph convolutional networks can effectively handle non-Euclidean spatial correlations and extract spatiotemporal features, some studies have incorporated graph convolutional networks into their models, such as graph attention convolutional networks (GACN), dynamic graph convolutional recursive filling networks (DGCRIN), and layered spatiotemporal graph convolutional networks (HSTGCN). However, the above methods cannot extract complete spatiotemporal correlations from traffic data with missing values, so their performance is unsatisfactory. Moreover, when the data missing rate is high, the defects of these methods will be further amplified. Summary of the invention

[0005] The present invention provides a data completion method, system, electronic device and medium based on spatiotemporal attention, which are used to solve the problem of poor performance of the data completion method.

[0006] According to one aspect of the present invention, a data completion method based on spatiotemporal attention is provided, comprising:

[0007] The input incomplete traffic data is divided according to the time dimension, and the divided data includes traffic data of multiple time segments;

[0008] Filling missing values ​​in the traffic data based on data currently collected by the sensor;

[0009] The first time segment data of the filled traffic data is used as input for training the data completion model;

[0010] S1: Extracting latent spatial features of input data based on the constructed multi-head self-attention network, the latent spatial features are used to characterize the correlation between traffic data collected by sensors in each time segment;

[0011] S2: converting the sensor topology in the real space into an adjacency matrix, and extracting the real space features of the adjacency matrix based on the constructed graph attention feedforward network, wherein the real space features are used to characterize the correlation between sensors in the real space;

[0012] S3: performing feature concatenation of the latent spatial feature and the real spatial feature to obtain a complete spatial feature of the first time segment;

[0013] S4: extracting the temporal correlation of the input data based on the constructed temporal attention module; the input of the temporal attention module is the complete spatial features of the first time segment and the traffic data in the second time segment, and the output is the complete spatiotemporal features of the first time segment;

[0014] S5: Inputting the complete spatiotemporal features of the first time segment into the multi-head self-attention network to extract latent spatial features, and obtaining a data completion result;

[0015] S6: re-execute steps S1-S5 to obtain an optimized data completion result;

[0016] Repeat steps S1-S6 until the preset iteration stop condition is met, and then output the final data completion result.

[0017] According to another aspect of the present invention, a data completion device based on spatiotemporal attention is provided, comprising:

[0018] A data partitioning unit, used to partition the input incomplete traffic data according to the time dimension, and the partitioned data includes traffic data of multiple time segments;

[0019] A data filling unit, used to fill missing values ​​in the traffic data based on data currently collected by the sensor;

[0020] An input unit, used to take the first time segment data of the filled traffic data as input for training the data completion model;

[0021] A latent space feature extraction unit, used to extract latent space features of input data based on the constructed multi-head self-attention network, wherein the latent space features are used to characterize the correlation between traffic data collected by the sensor in each time segment;

[0022] A real space feature extraction unit, used to convert the sensor topology in the real space into an adjacency matrix, and extract the real space features of the adjacency matrix based on the constructed graph attention feedforward network, wherein the real space features are used to characterize the correlation between sensors in the real space;

[0023] A concatenation unit, configured to concatenate the latent spatial feature with the real spatial feature to obtain a complete spatial feature of the first time segment;

[0024] A time feature extraction unit, used to extract the time correlation of input data based on the constructed time attention module; the input of the time attention module is the complete spatial feature of the first time segment and the traffic data in the second time segment, and the output is the complete spatiotemporal feature of the first time segment;

[0025] A preliminary data completion unit, configured to input the complete spatiotemporal features of the first time segment into the multi-head self-attention network to extract latent spatial features, and obtain a data completion result;

[0026] An optimized data completion unit is used to re-execute the steps in the latent space feature extraction unit, the real space feature extraction unit, the splicing unit, the time feature extraction unit and the preliminary data completion unit to obtain an optimized data completion result;

[0027] The iterative calculation unit is used to repeatedly execute the steps in the spatial feature extraction unit, the real spatial feature extraction unit, the splicing unit, the time feature extraction unit, the preliminary data completion unit and the optimized data completion unit until a preset iteration stop condition is met, and output the final data completion result.

[0028] According to another aspect of the present invention, an electronic device is provided, the electronic device comprising:

[0029] at least one processor; and

[0030] a memory communicatively connected to the at least one processor; wherein,

[0031] The memory stores a computer program that can be executed by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the data completion method based on spatiotemporal attention described in any embodiment of the present invention.

[0032] According to another aspect of the present invention, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the data completion method based on spatiotemporal attention described in any embodiment of the present invention when executed.

[0033] The technical solution of the embodiment of the present invention uses self-attention to construct a multi-head self-attention network to extract latent spatial features in traffic data, uses a graph attention layer and a feedforward neural network to construct a graph attention feedforward network to extract real spatial features in traffic data, and achieves comprehensive extraction of spatial features by fusing the two features; uses a temporal attention module to extract temporal correlation, and achieves comprehensive extraction of spatiotemporal features by extracting spatial features and temporal features in sequence. The present invention constructs a traffic data completion framework based on spatiotemporal attention, uses a completion optimization serial mode distribution to perform completion operations, and uses an iterative strategy to optimize the completion results, which can be applied to data completion tasks in a variety of data missing scenarios.

[0034] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present invention, nor are they intended to limit the scope of the present invention. Other features of the present invention will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0036] Figure 1 is a flow chart of a data completion method based on spatiotemporal attention provided according to Embodiment 1 of the present invention;

[0037] Figure 2 It is an architecture diagram of a data completion model of a data completion method based on spatiotemporal attention in an embodiment of the present invention;

[0038] Figure 3 A schematic diagram of traffic data collected in a random data loss mode and a simulated random power failure mode in an embodiment of the present invention;

[0039] FIG4( a ) is a schematic diagram of the MAE of the PEMSBAY dataset in a random data missing mode according to an embodiment of the present invention;

[0040] FIG4( b ) is a schematic diagram of the RMSE of the PEMSBAY dataset in a random data missing mode according to an embodiment of the present invention;

[0041] FIG4( c ) is a schematic diagram of the MAE of the PEMSBAY dataset in a simulated random power-off mode according to an embodiment of the present invention;

[0042] FIG4( d ) is a schematic diagram of the RMSE of the PEMSBAY dataset in a simulated random power-off mode according to an embodiment of the present invention;

[0043] FIG5( a ) is a schematic diagram of indicators of the PEMSBAY dataset in a random data missing mode according to an embodiment of the present invention;

[0044] FIG5( b ) is a schematic diagram of indicators of the PEMSBAY dataset in a simulated random power-off mode according to an embodiment of the present invention;

[0045] FIG5( c ) is a schematic diagram of indicators of the Seattle dataset in a random data missing mode according to an embodiment of the present invention;

[0046] FIG5( d ) is a schematic diagram of indicators of the Seattle data set in a simulated random power outage mode according to an embodiment of the present invention;

[0047] Figure 6 A schematic diagram showing how the completion effect is affected by loss function parameters in an embodiment of the present invention;

[0048] Figure 7 is a schematic diagram of the results of an ablation experiment performed in an embodiment of the present invention;

[0049] Figure 8 A flow chart of a data completion device based on spatiotemporal attention provided in an embodiment of the present invention;

[0050] Fig. 9 It is a structural schematic diagram of an electronic device for implementing the data completion method based on spatiotemporal attention according to an embodiment of the present invention. DETAILED DESCRIPTION

[0051] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0052] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0053] In recent years, a lot of research has been carried out at home and abroad on traffic data completion. Existing traffic data completion methods can be roughly divided into five categories: 1) prediction-based methods; 2) interpolation-based methods; 3) statistical learning-based methods; 4) matrix / tensor completion methods; 5) deep learning methods.

[0054] The prediction-based method is to build a prediction model based on historical data to predict missing data. Linear regression (LR), support vector regression (SVR), autoregressive integrated moving average (ARIMA) model and Bayesian network (BNs) all belong to this category. However, the above methods are highly dependent on data quality. If there is a lot of noise in the data, the constructed prediction model will be seriously affected. In addition, these methods lack sufficient expression of complex correlations. For example, although Bayesian networks can represent the causal relationship between variables to a certain extent, the expression of complex and dynamic spatiotemporal correlations in traffic scenarios is still insufficient.

[0055] Interpolation-based methods are historical average algorithms. This means that when dealing with missing data, they refer to some average situations in historical data for estimation. For example, K-nearest neighbor algorithm (K-NN) and local least squares (LLS), they use temporal proximity or pattern proximity to estimate missing data. The key difficulty of these methods is to use a suitable distance metric to identify neighbors. For example, when judging the similarity of two data points, whether to use Euclidean distance (the straight-line distance between the two points), Manhattan distance (the sum of the horizontal and vertical distances of the two points on the coordinate axis) or other distance metrics, choosing different distance metrics will lead to different neighbors found, which will have a significant impact on the final result of estimating missing data. In addition, such methods can only take into account the proximity of a single level, which is bound to ignore a lot of information.

[0056] Methods based on statistical learning include Markov Monte Carlo (MCMC), probabilistic principal component analysis (PPCA), and Bayesian principal component analysis (BPCA). Such methods first assume that the data follows a certain probability distribution, such as normal distribution, Poisson distribution, etc., which is the basic premise for subsequent analysis and processing. After that, they will use the existing observed data for iterative operations to estimate the missing data. However, due to the high instability of traffic data, it is difficult for methods based on statistical learning to assume an accurate probability distribution for traffic data, especially when the data missing rate is high. This is because there are relatively few observations available for analysis and inference, and it is difficult to reasonably determine the probability distribution form that the entire data set (including the missing part) should conform to based on limited observations, which seriously affects the accuracy of these statistical learning-based methods in estimating missing data.

[0057] The core of matrix / tensor completion methods is to use various matrix and tensor computing techniques to mine multidimensional spatiotemporal correlations in data, thereby realizing the inference of missing guidance. For example, Bayesian Time Matrix Factorization (BTMF) uses Gaussian Autoregression (VAR) process to model the time factor matrix. Through this modeling method, it is possible to better handle time-related data features so that the rules in the time dimension can be used to infer missing values ​​when completing the matrix. Bayesian Gaussian CP Factorization (BGCP) further extends the Bayesian probability matrix factorization model to the case of high-order tensors, specifically for data completion tasks. High-order tensors can handle more complex data structures. Through this extension, missing data can be attempted to be completed in more complex data scenarios. However, a large number of experiments have shown that different missing patterns have a great impact on the completion effect of matrix / tensor completion methods. For example, the results are poor under the completely random missing pattern (MCAR), which may be because these methods find it difficult to capture the characteristics of adjacent structures in the data under this pattern.

[0058] Based on this, the present invention provides a data completion method based on spatiotemporal attention, which uses self-attention to construct a multi-head self-attention network to extract potential spatial features in traffic data, uses a graph attention layer and a feedforward neural network to construct a graph attention feedforward network to extract real spatial features in traffic data, and integrates the two features to achieve comprehensive extraction of spatial features; uses a temporal attention module to extract temporal correlation, and achieves comprehensive extraction of spatiotemporal features by extracting spatial features and temporal features in sequence. The present invention constructs a traffic data completion framework based on spatiotemporal attention, uses a completion optimization serial mode distribution to perform completion operations, and uses an iterative strategy to optimize the completion results, which can be applied to data completion tasks in a variety of data missing scenarios.

[0059] Embodiment 1

[0060] Figure 1 A flow chart of a data completion method based on spatiotemporal attention is provided for the first embodiment of the present invention. Figure 1 As shown, the method includes:

[0061] S101. Divide the input incomplete traffic data according to the time dimension, and the divided data includes traffic data of multiple time segments.

[0062] It should be noted that the missing data in the collected traffic data is completely random and has nothing to do with the value of the data itself or the values ​​of other variables, that is, the location of missing data in the data collected in different time segments is not the same. In addition, each sensor is randomly powered off for a certain period of time, during which the sensor cannot collect data, and the selection of sensors is also completely random. Therefore, the data obtained in different time segments may be collected by different sensors.

[0063] Specifically, the time segment length F is used as the window length, and the initial traffic data X ori Divide into T groups and get the divided data in T=D / F, where D represents the total duration of traffic data, N represents the amount of data at each time point, and T represents the number of time segments contained in D.

[0064] Then the data in time segment t

[0065] S102: Fill in missing values ​​in the traffic data based on data currently collected by the sensor.

[0066] Since there are random missing data in the traffic data, there are certain missing values. Therefore, in this embodiment, a preset filling method can be used to supplement the missing values. For example, real-time data collected by sensors can be used to fill the missing values ​​in the initial traffic data.

[0067] S103: Using the first time segment data of the filled traffic data as input for training a data completion model.

[0068] The first time segment data may be traffic data of any time segment t, and the traffic data is traffic data after missing values ​​are filled in the initial traffic data, and is used as the input of the data completion model to iteratively train the data completion model.

[0069] S104. Extracting latent spatial features of the input data based on the constructed multi-head self-attention network, wherein the latent spatial features are used to characterize the correlation between the traffic data collected by the sensor in each time segment.

[0070] In this embodiment, a constructed multi-head self-attention network is used to extract the latent spatial features of the input data. The attention heads of the multi-head self-attention network may include multiple linear layers, and the attention mechanism is used to enable the model to focus on the key parts of the input data.

[0071] It should be noted that the attention mechanism calculates the relevance score based on the query vector and the key vector, and then converts the score into weights through the softmax function. Finally, these weights are used to perform weighted summation on the value vector to obtain the attention output, so that the model can focus on the key parts of the input data.

[0072] S105. Convert the sensor topology in the real space into an adjacency matrix, and extract the real space features of the adjacency matrix based on the constructed graph attention feedforward network, where the real space features are used to characterize the correlation between sensors in the real space.

[0073] Since traffic data is collected by sensors at different spatial locations, that is, the sensors used for the collected data have certain spatial correlation. This embodiment uses the adjacency matrix converted from the sensor topology in real space, and extracts the real space features of the adjacency matrix based on the constructed graph attention feedforward network. Among them, the graph attention feedforward network uses a combination of a graph attention network and a feedforward network to extract the real space features of the sensor space topology.

[0074] S106: Concatenate the latent spatial features with the real spatial features to obtain a complete spatial feature of the first time segment.

[0075] The complete spatial features of the first time segment can be obtained by feature splicing the latent spatial features and the real spatial features. The complete spatial features of the first time segment can represent the complete spatial features obtained in the process of preliminarily completing the traffic data of the first time segment, or can represent the complete spatial features of the first time segment obtained by extracting the latent spatial features and the real spatial features from the preliminary completion data results in the process of optimizing data completion, and feature splicing the latent spatial features and the real spatial features.

[0076] S107. Extracting the temporal correlation of input data based on the constructed temporal attention module; the input of the temporal attention module is the complete spatial features of the first time segment and the traffic data in the second time segment, and the output is the complete spatiotemporal features of the first time segment.

[0077] It should be noted that the time attention module uses the attention mechanism to focus on the time feature part of the input data. The input of the time attention module is the complete spatial feature of the first time segment and the traffic data in the second time segment.

[0078] The first time segment represents any time segment, and the second time segment represents a time segment different from the first time segment. In a preferred solution, the second time segment can be a time segment adjacent to the first time segment, so that the complete spatial characteristics of the first time segment and the time correlation between the traffic data in the second time segment can be better extracted. For example, a time segment is divided into one hour intervals, and if the first time segment is from 1 to 2 o'clock, then the second time segment can be from 0 to 1 o'clock, or from 2 to 3 o'clock.

[0079] In addition, this embodiment inputs the traffic data of the first time segment into the spatial attention module and the temporal attention module in sequence, extracts the spatial features and the temporal features in sequence, and outputs the complete spatiotemporal features of the first time segment.

[0080] S108. Input the complete spatiotemporal features of the first time segment into the multi-head self-attention network to extract latent spatial features and obtain data completion results.

[0081] After the complete spatiotemporal features of the first time segment are extracted in this embodiment, the complete spatiotemporal features of the first time segment can be input into the multi-head self-attention network to extract latent spatial features, and a preliminary data completion result of the traffic data of the first time segment can be obtained.

[0082] S109. Re-execute steps S104-S108 to obtain an optimized data completion result.

[0083] The preliminary data completion result is then input into a multi-head self-attention network, and the multi-head self-attention network is used to extract the latent space features of the preliminary data completion result. The sensor topological structure in the real space is converted into an adjacency matrix, and the real space features of the adjacency matrix are extracted based on the constructed graph attention feedforward network. The latent space features are feature concatenated with the real space features to obtain the complete spatial features of the first time segment, and the temporal correlation of the input data is extracted based on the constructed temporal attention module. The input of the temporal attention module is the complete spatial features of the first time segment and the traffic data in the second time segment, and the output is the complete spatiotemporal features of the first time segment. The complete spatiotemporal features of the first time segment are then input into the multi-head self-attention network to extract the latent space features, and the optimized data completion result is obtained.

[0084] The preset loss function can be used to calculate the loss in the initial data completion process and the loss in the optimization process, and adjust the parameters of the data completion model.

[0085] S110, repeating steps S104-S109 until a preset iteration stop condition is met, and outputting a final data completion result.

[0086] Re-input traffic data of another first time segment, train the data completion model, repeat the iteration multiple times until the parameters of the data completion model converge or the number of iterations reaches a preset iteration number threshold, then stop the iteration and output the trained data completion model.

[0087] The technical solution of the embodiment of the present invention uses self-attention to construct a multi-head self-attention network to extract latent spatial features in traffic data, uses a graph attention layer and a feedforward neural network to construct a graph attention feedforward network to extract real spatial features in traffic data, and achieves comprehensive extraction of spatial features by fusing the two features; uses a temporal attention module to extract temporal correlation, and achieves comprehensive extraction of spatiotemporal features by sequentially extracting spatial features and temporal features of traffic data and fusing the two features. The present invention constructs a traffic data completion framework based on spatiotemporal attention, uses a completion optimization serial mode distribution to perform completion operations, and uses an iterative strategy to optimize the completion results, which can be applied to data completion tasks in a variety of data missing scenarios.

[0088] Embodiment 2

[0089] Figure 2 A scheme structure diagram of another embodiment of a data completion method based on spatiotemporal attention provided in Embodiment 2 of the present invention, Figure 2 include:

[0090] S201 . Divide the input incomplete traffic data according to the time dimension, and the divided data includes traffic data of multiple time segments.

[0091] Taking the time segment length F as the window length, the initial traffic data X ori Divide into T groups and get the divided data in T = D / F, where D represents the total duration of traffic data, N represents the amount of data at each time point, and T represents the number of time segments contained in D. Then the data in time segment t

[0092] S202: Fill in missing values ​​of the traffic data based on an average value of data currently collected by the sensor.

[0093] In this embodiment, the data collected by the sensor at the current moment can be weighted and summed, and the weighted summed value can be used to fill the missing position in the traffic data. In a preferred solution, the average value of the data collected by the sensor at the current moment can be used to fill the missing value in the traffic data.

[0094] S203: Using the first time segment data of the filled traffic data as input for training a data completion model.

[0095] S204, extracting latent spatial features of input data based on the constructed multi-head self-attention network.

[0096] Specifically, step S204 includes:

[0097] S2041. Calculate the attention score of each attention head based on the input data, and generate a reconstructed network based on the attention score of each attention head, where the reconstructed network is used to characterize the correlation between sensors.

[0098] The road network composed of all sensor nodes can be obtained as a complete graph, the correlation between every two sensors in each time segment can be calculated, and the reconstructed road network can be generated.

[0099] Specifically, in this embodiment, according to Use three linear layers to respectively calculate the Query of the i-th attention head in the time segment t. Key Value

[0100]

[0101] The query object and value objects Multiply to get the attention score of the i-th attention head in time segment t Among them, FC is the fully connected layer.

[0102]

[0103] T represents the number of time segments contained in D; F is the window length of the time segment.

[0104] Based on the above formula, the attention scores of all attention heads can be calculated, that is, the reconstructed network is generated.

[0105] It should be noted that in the attention mechanism, Query represents the query object, Value represents the object being queried, and Key represents the key vector used to measure the degree of association between Query and other vectors. The attention mechanism determines how much attention to allocate when weighted aggregation is performed on the value by calculating the similarity between Query and Key. Specifically, by calculating the dot product of Query and Key, and then performing a standardized operation (such as the softmax function), weights representing the importance of different Keys are obtained, and then these weights are weighted and summed with the corresponding Value to obtain the final representation.

[0106] S2042: Perform a mask operation on the reconstructed network.

[0107] In order to make the current node obtain information from other nodes as much as possible, the diagonal elements in are masked.

[0108] The mask operations include:

[0109] First construct the mask matrix:

[0110] M diag =1-I diag

[0111] Among them, I diag Identity matrix, size, and attention score of the i-th attention head Consistent.

[0112] Use the mask matrix M diag right Perform mask operations.

[0113]

[0114] Among them, ← represents a mask operation.

[0115] Reconstructing the road network Contains h Where h is the number of attention heads.

[0116] S2043, performing a standardization operation on the reconstructed network after the mask operation to obtain the features of each attention head.

[0117] According to the attention score of the i-th attention head Get the embedding of the i-th attention head

[0118]

[0119] Among them, softmax() represents regularization calculation, and embedding refers to the mapping from discrete variables to continuous numerical space.

[0120] S2044. Merge the features of each attention head to generate a first merged feature.

[0121] Merge the multi-head attention embeddings to get the first merged feature

[0122]

[0123] In the formula, || represents the concatenation operation; FC is the fully connected layer; Represents the traffic data within time segment t.

[0124] S2045. Use the first merged features through a feedforward neural network with residual connection to obtain the latent space features of the input data.

[0125] It should be noted that residual connection is an important technology in deep learning. In a neural network, it directly adds the input of a layer to the transformed output of the layer, which can effectively solve the problems of gradient vanishing and gradient exploding during deep neural network training, making it easier to train deep models, and can enable the network to learn the residual information in the input signal, so that the model can converge quickly.

[0126] Merge the first feature Latent space embedding via a feed-forward neural network with residual connections

[0127]

[0128] FF MSAN represents a feed-forward neural network.

[0129] S205. Convert the sensor topology in the real space into an adjacency matrix, and extract the real space features of the adjacency matrix based on the constructed graph attention feedforward network, where the real space features are used to characterize the correlation between sensors in the real space.

[0130] S2051. Use the constructed graph attention layer to extract features between sensor nodes.

[0131] It should be noted that the graph attention network is a deep learning model for graph data, which can learn the relationship between nodes and the representation of nodes. The network introduces an attention mechanism, which gives different weights to the relationship between nodes, so that the model can automatically focus on the nodes that are meaningful to the current task.

[0132] S2052. Input the features between the sensor nodes into a feedforward neural network with residual connection to obtain real space features.

[0133] The sensor topology in the real space is converted into the adjacency matrix Adj.

[0134] The graph attention mechanism is used to extract the real space correlation from the adjacency matrix Adj to obtain the real space embedding

[0135] Specifically, according to the adjacency matrix Adj, the attention coefficient between adjacent sensor nodes i and j is calculated:

[0136] e ij =LeakyReLU(a T [Wx t,i ||Wx t,j ])

[0137] where x t,i and x t,j is the embedding of nodes i and j in time segment t, W is the embedding conversion weight matrix, a is the weight parameter; ReLU is the activation function.

[0138] To make the attention coefficient easier to calculate and compare, the softmax function is used for regularization:

[0139] a ij =softmax(e ij )

[0140] The new embedding of node i is calculated as follows:

[0141]

[0142] where σ is a nonlinear activation function.

[0143] After all nodes are embedded and updated, the graph attention layer ends. The ReLU activation function is used between the two graph attention layers. The final embedding of the graph attention layer is X t '.

[0144] Embed the output of the graph attention layer into X t 'Input into a feedforward neural network with residual connection to obtain the real space features

[0145]

[0146] In the formula, FF GAT represents a feed-forward neural network.

[0147] S206: Input the latent space features and the real space features into the constructed multi-layer perceptron for feature fusion to obtain the complete space features of the first time segment.

[0148] The latent space features and real space characteristics Perform feature splicing and use multi-layer perceptron for feature fusion to obtain complete spatial embedding

[0149] Get the latent space embedding and real space embedding Finally, the two features are concatenated and fused using a multi-layer perceptron to obtain the complete spatial embedding of the time segment t

[0150]

[0151] MLP SAB Multilayer Perceptron computation representing the spatial self-attention module.

[0152] S207. Extract the time correlation of input data in the node dimension based on the constructed multi-head cross-attention network, wherein the input of the multi-head cross-attention network is the complete spatial features of the first time segment and the traffic data in the second time segment, and the output is the time features of the node dimension.

[0153] In one embodiment, step S207 includes:

[0154] S2071. Use at least one linear layer to process the complete spatial features of the first time segment, and use at least one linear layer to process the traffic data in the second time segment to obtain features of each attention head.

[0155] S2072, merge the features of each attention head to obtain the second merged feature.

[0156] S2073. Use the second merged features through a feedforward neural network with residual connections to obtain node dimension time features.

[0157] Specifically, two linear layers are used to process the traffic data of adjacent time segments. Processing, using a linear layer on Processing, get the Query of the i-th attention head Key Value

[0158]

[0159] according to Calculate the characteristics of the i-th attention head

[0160]

[0161] Merge the multi-head attention embedding to get the second merged feature

[0162]

[0163] The second merge feature Node-dimensional temporal features are obtained through a feed-forward neural network with residual connections

[0164]

[0165] Where FF represents a feedforward neural network with multi-head cross attention network.

[0166] S208. Extract the time correlation of the input data in the time dimension based on the constructed multi-head convolutional attention network, the input of the multi-head convolutional attention network is the complete spatial features of the first time segment and the traffic data in the second time segment, and the output is the time features of the time dimension.

[0167] Specifically, step S208 includes:

[0168] S2081, splicing the complete spatial features of the first time segment and the traffic data in the second time segment and adding position codes to obtain splicing features;

[0169] S2082, performing a filling operation on the splicing feature to obtain a filling feature;

[0170] S2083, calculating the features of each attention head based on the splicing features and the filling features;

[0171] S2084, merging the features of each attention head to obtain a third merged feature;

[0172] S2085. Pass the third merged feature through a feedforward neural network with residual connection to obtain a time dimension time feature.

[0173] Specifically, the complete spatial features of the first time segment and traffic data in the second time segment Perform splicing and add position coding to obtain splicing features

[0174]

[0175] For splicing features Perform the filling operation to obtain the filling feature

[0176]

[0177] Where k is the convolution kernel size, p=k-1, v=0 means the padding length is k-1 and the padding value is 0;

[0178] Use two convolution layers with kernel k as Processing, using a convolution layer with a convolution kernel of 1 to concatenate features Process it and get the Query of the first attention head Key Value

[0179]

[0180] according to Calculate the characteristics of the i-th attention head:

[0181]

[0182] Merge the multi-head attention embedding to get the third merged feature

[0183]

[0184] The third merge feature The time dimension time features are obtained through a feedforward neural network with residual connections:

[0185]

[0186] FF CSAN Represents the feedforward neural network computation of a multi-head convolutional attention network.

[0187] S209: Concatenate the time features of the node dimension and the time features of the time dimension to obtain complete spatiotemporal features of the first time segment.

[0188] Get node dimension time characteristics and time dimension time feature Finally, the two embeddings are concatenated and fused using a multi-layer perceptron to obtain the complete spatiotemporal features of time segment t

[0189]

[0190] MLP TAB Computation of the multilayer perceptron with the temporal attention module.

[0191] S210: Input the complete spatiotemporal features of the first time segment into the multi-head self-attention network to extract latent spatial features and obtain data completion results.

[0192] Specifically, it includes: obtaining the reconstruction network of the complete spatiotemporal features of time segment t in turn; performing mask operations on the reconstruction network; and performing standardization operations on the reconstruction network after the mask operation to obtain the features of each attention head. The features of each attention head are merged to generate merged features. The merged features are passed through a feedforward neural network with residual connections to obtain the latent spatial features of the input data and obtain preliminary data completion results. Specifically, Figure 2 The spatial multilayer perceptron in , which uses a multi-head self-attention network to extract the latent spatial features of the complete spatiotemporal features of the first time segment.

[0193] S211. Re-execute steps S204-S210 to obtain an optimized data completion result.

[0194] The preliminary data completion result is then input into a multi-head self-attention network, and the multi-head self-attention network is used to extract the latent space features of the preliminary data completion result. The sensor topological structure in the real space is converted into an adjacency matrix, and the real space features of the adjacency matrix are extracted based on the constructed graph attention feedforward network. The latent space features are feature concatenated with the real space features to obtain the complete spatial features of the first time segment, and the temporal correlation of the input data is extracted based on the constructed temporal attention module. The input of the temporal attention module is the complete spatial features of the first time segment and the traffic data in the second time segment, and the output is the complete spatiotemporal features of the first time segment. The complete spatiotemporal features of the first time segment are then input into the multi-head self-attention network to extract the latent space features, and the optimized data completion result is obtained.

[0195] S212: Repeat steps S204-S211 until a preset iteration stop condition is met, and output the final data completion result.

[0196] Re-input traffic data of another first time segment, train the data completion model, repeat the iteration multiple times until the parameters of the data completion model converge or the number of iterations reaches a preset iteration number threshold, then stop the iteration and output the trained data completion model.

[0197] In this embodiment, the completion optimization loss function L is calculated as follows:

[0198]

[0199] In the formula, represents the mean absolute error loss of the initial completion process, Represents the mean absolute error loss of the optimization completion process. Of course, the loss function in this embodiment can also select the root mean square error loss, or other loss functions.

[0200] In the above formula, α controls the weights of the two network losses in the initial completion process and the optimized completion process, such as Figure 6 Schematic diagram showing how the completion effect is affected by the loss function parameter α.

[0201] In a specific application example, the present invention can use traffic data collected by multiple sensor nodes within half a year as initial traffic data, divide the initial sample data into time segments of 5 minutes, and obtain a large number of samples. Figure 3 The left figure is a traffic data diagram under the random missing data mode. Figure 3 The figure on the right is a traffic data graph in the sensor random power-off mode. The present invention uses the traffic data graph in the data random missing mode for comparative experiments, and compares the mean absolute error, root mean square error and mean absolute percentage error of the traffic data completion method STAIRN based on spatiotemporal attention of the present invention with other schemes (BGCP, DSAE, MIWAE, GA-GAN, HSTGCN). The comparison results are shown in Table 1 below:

[0202] Table 1 Comparison of MAE, RMSE and MAPE indicators in the random missing data mode

[0203]

[0204] It can be seen from Table 1 above that the performance indicators of the present invention are better than all other comparison models on both the PEMSBAY dataset and the Seattle dataset, proving the effectiveness of the present invention. This is because the present invention can extract comprehensive spatiotemporal features from partially observed traffic data for data completion. Among the comparison models, the completion method based on graph sampling aggregation and generative adversarial networks achieved the best results, because this method can comprehensively extract spatial correlations in traffic data and infer missing values ​​through generative adversarial networks. However, this method ignores the temporal features, so the effect is worse than that of the present invention. The completion method based on layered graph convolution can extract fine-grained spatiotemporal information, but the extracted information is not comprehensive, especially ignoring the temporal features of the node dimension, resulting in poor results of this method. The effects of all comparison models drop sharply when the missing rate is high, because these methods cannot extract spatiotemporal features from limited data, while the present invention can effectively capture comprehensive spatiotemporal features from sparse traffic data through specially designed spatial and temporal attention modules, so it maintains excellent completion effects when the missing rate is high.

[0205] Table 2 shows the results of the comparative experiment under the simulated random power-off mode. It can be seen from the experimental results that the performance indicators of the present invention on the two data sets are better than those of all other comparative models, proving the effectiveness of the present invention. This is because the present invention can extract comprehensive spatiotemporal features from partially observed traffic data for data completion. Among the comparative models, the completion method based on layered graph convolution has the best effect when the missing rate is low. This is because the macro layer and road layer in this method can capture fine-grained spatiotemporal features from sufficient data. The matrix completion method based on Gaussian vector autoregression has also achieved good results. This is because this method uses the Gaussian vector autoregression process to model the time factor matrix and can handle time continuous missing very well. Similarly, the effects of all comparative models drop sharply at high missing rates, while the present invention can maintain excellent completion effects when the missing rate is high, which once again proves the effectiveness of the technical solution in the present invention.

[0206] Table 2 Comparison of MAE, RMSE and MAPE indicators under random power outage mode

[0207]

[0208] Figure 4(a) is a schematic diagram of the MAE of the traffic data completion method STAIRN based on spatiotemporal attention of the present invention and other schemes (BTMF, GA-GAN) under the random data missing mode of PEMSBAY dataset; Figure 4(b) is a schematic diagram of the RMSE of the traffic data completion method STAIRN based on spatiotemporal attention of the present invention and other schemes (BTMF, GA-GAN) under the random data missing mode of PEMSBAY dataset; Figure 4(c) is a schematic diagram of the MAE of the traffic data completion method STAIRN based on spatiotemporal attention of the present invention and other schemes (BTMF, GA-GAN) under the simulated random power outage mode of PEMSBAY dataset; Figure 4(d) is a schematic diagram of the RMSE of the traffic data completion method STAIRN based on spatiotemporal attention of the present invention and other schemes (BTMF, GA-GAN) under the simulated random power outage mode of PEMSBAY dataset. Figure 4(a)-Figure 4(d) It can be seen that the solutions of the present invention are superior to other solutions.

[0209] Figure 5(a) is a schematic diagram of the indicators of the PEMSBAY dataset in the random data missing mode; Figure 5(b) is a schematic diagram of the indicators of the PEMSBAY dataset in the simulated random power outage mode; Figure 5(c) is a schematic diagram of the indicators of the Seattle dataset in the random data missing mode; Figure 5(d) is a schematic diagram of the indicators of the Seattle dataset in the simulated random power outage mode.

[0210] The effectiveness of each module of the present invention is verified by ablation experiments. Figure 7The schematic diagram of the ablation experiment results is shown in Figure 1. Figure 7 In the paper, a simple network IRN containing only a simple multilayer perceptron is used as the base network, and its root mean square error (RMSE) is 2.21. Next, the effects of these modules are studied by gradually integrating the proposed modules into the base network. (2) CAIRN uses a multi-head cross-attention network in the temporal attention module to extract temporal features between different sensors, and achieves good improvement, which proves that the rich temporal features between sensors can greatly help the model reconstruct missing values. Subsequently, by further introducing a convolutional attention network to extract temporal features within the sensor, TAIRN improves the RMSE to 1.97. This shows that the two modules extract temporal features from different perspectives, and through their collaboration, they can capture comprehensive temporal information from traffic data. (3) GTAIRN uses a graph attention feedforward network to obtain real spatial features from the road network structure, and then STAIRN introduces a multi-head self-attention network on top of it to utilize the latent space features between detectors, which reduces the RMSE to 1.91. This means that the spatial features not only cover the geographical proximity in the real road network structure, but also the proximity in the latent space. In summary, the RMSE was gradually optimized from 2.23 to 1.91, and the performance was improved by 14.3%.

[0211] The embodiment of the present invention uses self-attention to construct a multi-head self-attention network to extract latent spatial features in traffic data, uses a graph attention layer and a feedforward neural network to construct a graph attention feedforward network to extract real spatial features in traffic data, and fuses the two features to achieve comprehensive extraction of spatial features; uses cross-attention to construct a multi-head cross-attention network to extract the time correlation of the node dimension, and uses attention and convolution to construct a multi-head convolutional attention network to extract the time correlation of the time dimension. By fusing the two features, comprehensive extraction of time features is achieved. The present invention constructs a traffic data completion framework based on spatiotemporal attention, adopts completion optimization serial mode distribution to perform completion operations, and uses an iterative strategy to optimize the completion results. It can be applied to data completion tasks in a variety of data missing scenarios.

[0212] Embodiment 3

[0213] Figure 8 This is a schematic diagram of the structure of a data completion device based on spatiotemporal attention provided in the third embodiment of the present invention. Figure 8 As shown, the device comprises:

[0214] The data division unit 401 is used to divide the input incomplete traffic data according to the time dimension, and the divided data includes traffic data of multiple time segments;

[0215] A data filling unit 402, configured to fill missing values ​​in the traffic data based on data currently collected by the sensor;

[0216] An input unit 403, used to take the first time segment data of the filled traffic data as input for training the data completion model;

[0217] A latent space feature extraction unit 404 is used to extract latent space features of input data based on the constructed multi-head self-attention network, wherein the latent space features are used to characterize the correlation between the traffic data collected by the sensor in each time segment;

[0218] A real space feature extraction unit 405 is used to convert the sensor topology in the real space into an adjacency matrix, and extract the real space features of the adjacency matrix based on the constructed graph attention feedforward network, wherein the real space features are used to characterize the correlation between sensors in the real space;

[0219] A splicing unit 406 is used to perform feature splicing on the latent spatial feature and the real spatial feature to obtain a complete spatial feature of the first time segment;

[0220] A time feature extraction unit 407 is used to extract the time correlation of the input data based on the constructed time attention module; the input of the time attention module is the complete spatial feature of the first time segment and the traffic data in the second time segment, and the output is the complete spatiotemporal feature of the first time segment;

[0221] A preliminary data completion unit 408 is used to input the complete spatiotemporal features of the first time segment into the multi-head self-attention network to extract latent spatial features, and obtain a data completion result;

[0222] The optimized data completion unit 409 is used to re-execute the steps in the latent space feature extraction unit, the real space feature extraction unit, the splicing unit, the time feature extraction unit and the preliminary data completion unit to obtain an optimized data completion result;

[0223] The iterative calculation unit 410 is used to repeatedly execute the steps in the spatial feature extraction unit, the real spatial feature extraction unit, the splicing unit, the time feature extraction unit, the preliminary data completion unit and the optimized data completion unit until a preset iteration stop condition is met, and output the final data completion result.

[0224] The message data processing resource scheduling device provided in the embodiment of the present invention can execute the message data processing resource scheduling device method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0225] Embodiment 4

[0226] Fig. 9 A schematic diagram of the structure of an electronic device 10 that can be used to implement an embodiment of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices (such as helmets, glasses, watches, etc.) and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or required herein.

[0227] like Fig. 9 As shown, the electronic device 10 includes at least one processor 11, and a memory connected to the at least one processor 11, such as a read-only memory (ROM) 12, a random access memory (RAM) 13, etc., wherein the memory stores a computer program that can be executed by at least one processor, and the processor 11 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 12 or the computer program loaded from the storage unit 18 to the random access memory (RAM) 13. In the RAM 13, various programs and data required for the operation of the electronic device 10 can also be stored. The processor 11, the ROM 12, and the RAM 13 are connected to each other through a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0228] A number of components in the electronic device 10 are connected to the I / O interface 15, including: an input unit 16, such as a keyboard, a mouse, etc.; an output unit 17, such as various types of displays, speakers, etc.; a storage unit 18, such as a disk, an optical disk, etc.; and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0229] The processor 11 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The processor 11 executes the various methods and processes described above, such as a data completion method based on spatiotemporal attention.

[0230] In some embodiments, a method for data completion based on spatiotemporal attention may be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as a storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 10 via the ROM 12 and / or the communication unit 19. When the computer program is loaded into the RAM 13 and executed by the processor 11, one or more steps of the method for data completion based on spatiotemporal attention described above may be performed. Alternatively, in other embodiments, the processor 11 may be configured to perform a method for data completion based on spatiotemporal attention in any other appropriate manner (e.g., by means of firmware).

[0231] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps described in the present invention can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solution of the present invention can be achieved, and this document does not limit this.

[0232] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A data completion method based on spatiotemporal attention, characterized in that: include: The input incomplete traffic data is divided according to the time dimension, and the divided data includes traffic data of multiple time segments; Filling missing values ​​in the traffic data based on data currently collected by the sensor; The first time segment data of the filled traffic data is used as input for training the data completion model; S1: Extracting latent spatial features of input data based on the constructed multi-head self-attention network, the latent spatial features are used to characterize the correlation between traffic data collected by sensors in each time segment; S2: converting the sensor topology in the real space into an adjacency matrix, and extracting the real space features of the adjacency matrix based on the constructed graph attention feedforward network, wherein the real space features are used to characterize the correlation between sensors in the real space; S3: performing feature concatenation of the latent spatial feature and the real spatial feature to obtain a complete spatial feature of the first time segment; S4: extracting the temporal correlation of the input data based on the constructed temporal attention module; the input of the temporal attention module is the complete spatial features of the first time segment and the traffic data in the second time segment, and the output is the complete spatiotemporal features of the first time segment; S5: Inputting the complete spatiotemporal features of the first time segment into the multi-head self-attention network to extract latent spatial features, and obtaining a data completion result; S6: re-execute steps S1-S5 to obtain an optimized data completion result; Repeat steps S1-S6 until the preset iteration stop condition is met, and then output the final data completion result.

2. The data completion method based on spatiotemporal attention according to claim 1, characterized in that: The filling of missing values ​​in the traffic data based on the data currently collected by the sensor includes: The missing values ​​of the traffic data are filled based on the average value of the data currently collected by the sensor.

3. The data completion method based on spatiotemporal attention according to claim 1, characterized in that: The multi-head self-attention network constructed based on the extraction of latent spatial features of input data includes: Calculate the attention score of each attention head based on the input data, and generate a reconstruction network based on the attention score of each attention head, wherein the reconstruction network is used to characterize the correlation between sensors; Performing a mask operation on the reconstructed network; Performing a standardization operation on the reconstructed network after the mask operation to obtain the features of each attention head; Merge the features of each attention head to generate the first merged feature; The first merged features are passed through a feedforward neural network with residual connections to obtain the latent space features of the input data.

4. The data completion method based on spatiotemporal attention according to claim 1, characterized in that: The real spatial features of the adjacency matrix are extracted based on the constructed graph attention feedforward network, including: The constructed graph attention layer is used to extract features between sensor nodes; The features between sensor nodes are input into a feed-forward neural network with residual connections to obtain the real space features.

5. The data completion method based on spatiotemporal attention according to claim 1, characterized in that: The step of performing feature concatenation of the latent spatial feature and the real spatial feature to obtain a complete spatial feature of the first time segment includes: The latent spatial features and the real spatial features are input into the constructed multi-layer perceptron for feature fusion to obtain the complete spatial features of the first time segment.

6. The data completion method based on spatiotemporal attention according to claim 1, characterized in that: The constructed time attention module extracts the time correlation of the input data; the input of the time attention module is the complete spatial features of the first time segment and the traffic data in the second time segment, and the output is the complete spatiotemporal features of the first time segment, including: Extracting the temporal correlation of input data in the node dimension based on the constructed multi-head cross attention network, wherein the input of the multi-head cross attention network is the complete spatial features of the first time segment and the traffic data in the second time segment, and the output is the temporal features of the node dimension; Extracting the time correlation of input data in the time dimension based on the constructed multi-head convolutional attention network, wherein the input of the multi-head convolutional attention network is the complete spatial features of the first time segment and the traffic data in the second time segment, and the output is the time features of the time dimension; The time features of the node dimension and the time features of the time dimension are concatenated to obtain complete spatiotemporal features of the first time segment.

7. The data completion method based on spatiotemporal attention according to claim 6, characterized in that: The multi-head cross attention network constructed based on the above method extracts the temporal correlation of the input data in the node dimension, including: Using at least one linear layer to process the complete spatial features of the first time segment, and using at least one linear layer to process the traffic data in the second time segment to obtain features of each attention head; Merge the features of each attention head to obtain the second merged feature; The second merged feature is passed through a feedforward neural network with residual connection to obtain the node dimension time feature.

8. The data completion method based on spatiotemporal attention according to claim 6, characterized in that: The multi-head convolutional attention network constructed based on the above method extracts the temporal correlation of the input data in the time dimension, including: splicing the complete spatial features of the first time segment and the traffic data in the second time segment and adding position coding to obtain splicing features; Performing a filling operation on the splicing feature to obtain a filling feature; Calculate the features of each attention head based on the splicing features and the filling features; Merge the features of each attention head to obtain the third merged feature; The third merged feature is passed through a feedforward neural network with a residual connection to obtain a time dimension time feature.

9. The data completion method based on spatiotemporal attention according to claim 1, characterized in that: The loss function of the data completion model is: In the formula, α controls the weights of the two network losses in the initial completion process and the optimized completion process; represents the loss function of the initial completion process, Represents the loss function of the optimization completion process.

10. A data completion device based on spatiotemporal attention, characterized in that: include: A data partitioning unit, used to partition the input incomplete traffic data according to the time dimension, and the partitioned data includes traffic data of multiple time segments; A data filling unit, used to fill missing values ​​in the traffic data based on data currently collected by the sensor; An input unit, used to take the first time segment data of the filled traffic data as input for training the data completion model; A latent space feature extraction unit, used to extract latent space features of input data based on the constructed multi-head self-attention network, wherein the latent space features are used to characterize the correlation between traffic data collected by the sensor in each time segment; A real space feature extraction unit, used to convert the sensor topology in the real space into an adjacency matrix, and extract the real space features of the adjacency matrix based on the constructed graph attention feedforward network, wherein the real space features are used to characterize the correlation between sensors in the real space; A concatenation unit, configured to concatenate the latent spatial feature with the real spatial feature to obtain a complete spatial feature of the first time segment; A time feature extraction unit, used to extract the time correlation of input data based on the constructed time attention module; the input of the time attention module is the complete spatial feature of the first time segment and the traffic data in the second time segment, and the output is the complete spatiotemporal feature of the first time segment; A preliminary data completion unit, configured to input the complete spatiotemporal features of the first time segment into the multi-head self-attention network to extract latent spatial features, and obtain a data completion result; An optimized data completion unit is used to re-execute the steps in the latent space feature extraction unit, the real space feature extraction unit, the splicing unit, the time feature extraction unit and the preliminary data completion unit to obtain an optimized data completion result; The iterative calculation unit is used to repeatedly execute the steps in the spatial feature extraction unit, the real spatial feature extraction unit, the splicing unit, the time feature extraction unit, the preliminary data completion unit and the optimized data completion unit until a preset iteration stop condition is met, and output the final data completion result.

Images