Graph neural network real-time OD matrix completion method based on flow conservation constraint

Abstract

The invention provides a graph neural network real-time OD matrix completion method based on flow conservation constraint, relates to the field of traffic system data completion and prediction, and solves the problem that an OD matrix is incomplete during real-time traffic flow prediction in the prior art. The method comprises the following steps: carrying out preliminary estimation on an incomplete real-time OD matrix by using GAT, and generating preliminary OD matrix estimation as a pre-interpolation result after feature transmission; acquiring historical observation data, capturing a space-time dependency relationship of the historical observation data through a mode of combining GCN and GRU, generating historical OD flow prediction, and taking the historical OD flow prediction as a historical accumulation result; and according to the real-time originating traffic, the pre-interpolation result and the historical accumulation result, traffic distribution is carried out by applying a residual learning mode, and a completed final OD matrix is generated under the condition of meeting traffic conservation constraints. According to the method, an efficient and accurate real-time OD matrix completion effect can be realized.

Description

Technical Field

[0001] The present invention relates to the field of traffic system data completion and prediction, and is applied to the prediction and management of traffic flow in intelligent transportation. Specifically, it relates to a real-time OD matrix completion method based on a graph neural network with flow conservation constraints. Background Art

[0002] In the traffic flow prediction and management of a traffic network, the OD matrix (Origin-Destination Matrix) is one of the core data, which is widely used in fields such as congestion prediction, route optimization, and travel time prediction. The OD matrix records the traffic flow between each origin and destination within a specific time window. It can accurately reflect the spatial and temporal distribution of traffic demand and is the basis for analyzing traffic flow patterns. Existing OD matrix prediction methods mainly rely on historical data and real-time observation data, aiming to predict traffic flow in different time periods. However, in existing methods, it is generally assumed that the OD matrix is complete and applicable to scenarios where the departure and arrival points are known, such as online car-hailing and taxi services. In contrast, for transportation modes such as subways and shared bicycles, the destinations of passengers are usually only confirmed when they arrive, which leads to the problem of OD matrix incompleteness.

[0003] To address the real-time incompleteness problem of the OD matrix, some studies have adopted compensation strategies to handle missing data. For example: handling incomplete subway data through a masking strategy and using non-negative matrix factorization (NMF) to infer the attributes of OD pairs; proposing a high-order weighted dynamic mode decomposition (HW-DMD) model and using boarding demand as alternative data to fill in the missing OD flow; introducing a heterogeneous information aggregation mechanism (HIAM) and learning the evolution of OD patterns by fusing incomplete OD matrices, complete departure data, and historical data. However, these methods all rely on additional data input and complex preprocessing processes and cannot directly solve the problem of missing real-time OD matrices.

[0004] Therefore, the existing technologies have shown the following key problems in multiple aspects. These problems not only limit the accuracy of real-time OD matrix estimation but also have a significant impact on the effectiveness of traffic flow prediction and management:

[0005] 1. Incompleteness of real-time OD matrix: In modern urban transportation systems, the acquisition of real-time OD matrix is ​​limited by the data collection method and time window. Especially in public transportation systems such as subways and shared bicycles, the destination information of passengers is often not available at the time of departure, and some trips may span multiple time periods, resulting in the inability to fully record in a short period of time. Data collection methods such as smart cards only update data when passengers arrive at the destination. Therefore, passenger data at different stages of the journey will be omitted or delayed, forming an incomplete OD matrix. This real-time missing data not only affects the accurate reflection of traffic demand, but also poses great challenges to subsequent traffic flow prediction and management decisions. Although existing technologies have tried to supplement missing data through various strategies (such as historical data interpolation, non-negative matrix decomposition, etc.), these methods still rely on certain known supplementary information and cannot directly address the problem of missing real-time OD matrix. How to restore a complete and accurate matrix from a partially missing real-time OD matrix is ​​still a technical problem that needs to be solved.

[0006] 2. Insufficient modeling of spatiotemporal dependencies: The importance of the OD matrix in traffic flow prediction lies not only in its capture of the spatial distribution of traffic demand, but also in its changes in the temporal dimension. Traditional OD matrix completion methods are often based on inferences from historical data, ignoring the spatiotemporal dependencies in real-time data. Traffic demand is highly spatiotemporally correlated, and is affected not only by spatial structure but also by dynamic changes in time. For example, the flow between subway stations will not only change over time, but also by external factors such as events and weather. Therefore, when estimating the OD matrix in real time, how to effectively capture these spatiotemporal dependencies at the same time will become the key to improving the accuracy of estimation. Most existing OD matrix prediction methods usually adopt static or simple time series-based models, which fail to fully utilize the complex interactions between time and space in the OD matrix. How to design a model that can simultaneously consider spatiotemporal dependencies, dynamic changes, and has sufficient flexibility is the key to improving the prediction accuracy of the OD matrix.

[0007] 3. Insufficient application of flow conservation constraints: In traffic flow prediction, flow conservation is a fundamental physical constraint, meaning that the departure flow at each station should be equal to the sum of all arrival flows. Most current OD matrix prediction methods do not fully consider this physical constraint during the completion process, resulting in a situation where the departure flow and arrival flow at some stations in the completed OD matrix may not match. This not only violates the basic principles of traffic flow but also leads to inaccurate flow prediction results, thereby affecting subsequent traffic planning and decision-making. How to effectively embed the flow conservation constraint into the real-time OD matrix completion process and ensure the consistency of departure flow and arrival flow at each station is an important issue overlooked by existing methods. Simple completion methods cannot guarantee this physical consistency and may result in the completed OD matrix not being able to truly reflect the actual situation of traffic flow.

[0008] 4. Handling the real-time and computational efficiency issues of large-scale traffic networks: The traffic networks in modern cities are large in scale and the traffic flow is extremely complex. Real-time processing and prediction of large OD matrices require efficient computational methods. However, although some existing methods can handle a certain amount of data, as the network scale expands, the computational complexity and storage requirements also increase significantly. How to reduce the computational complexity and improve the real-time performance of the model while ensuring high accuracy, especially for real-time OD matrix completion in large-scale traffic networks, is another challenge faced by current technologies. Many existing OD matrix prediction methods face computational bottlenecks when dealing with large-scale traffic data. Especially when it is necessary to handle spatio-temporal dependencies and physical constraints simultaneously, the complexity and computational volume of the model increase sharply. Therefore, how to optimize the model, reduce the computational complexity, and ensure real-time performance is a difficult problem that cannot be effectively solved by existing technologies. Summary of the Invention

[0009] Based on the current situation in the background technology, the purpose of the present invention is to solve the problem of incomplete OD matrices in existing technologies for real-time traffic flow prediction. Therefore, a real-time OD matrix completion method based on a graph neural network with flow conservation constraints is proposed. The method of the present invention combines a graph neural network, historical data, and flow conservation constraints to achieve efficient and accurate real-time OD matrix completion.

[0010] The present invention adopts the following technical solutions to achieve the purpose:

[0011] A real-time OD matrix completion method based on a graph neural network with flow conservation constraints, the method comprising the following steps:

[0012] S1. Pre-imputation: Use the Graph Attention Network (GAT) to perform a preliminary estimation on the incomplete real-time OD matrix, capture the spatial dependence relationship between each node, and generate a preliminary OD matrix estimate after feature transfer as the pre-imputation result;

[0013] S2. Historical Cumulation: Obtain historical observation data, and capture the spatio-temporal dependence relationship of the historical observation data through the combination of the graph convolutional network (GCN) and the gated recurrent unit (GRU) to generate a historical OD flow prediction as the historical cumulation result.

[0014] S3. OD Allocation: Obtain the real-time origin flow, and perform flow allocation based on the real-time origin flow, the pre-interpolation result, and the historical cumulation result through the application of residual learning to generate the final completed OD matrix under the condition of satisfying the flow conservation constraint.

[0015] Furthermore, in step S1, the incomplete real-time OD matrix is denoted as X t , representing the observed value with the target time interval t, and this observed value is used as the input of the graph attention network (GAT); the output of the graph attention network GAT is denoted as , representing the generated preliminary OD matrix estimate; the construction of the graph attention network GAT is completed by constructing a binary adjacency matrix.

[0016] Specifically, in the graph attention network GAT, the node feature matrix where N is the number of nodes, l is the number of layers, and F (l) is the feature dimension of the l-th layer, represents the set of real numbers; the real-time OD matrix X t is used as the initial feature matrix H (0) , that is, H (0) =X t ; the process of preliminary estimation to generate the preliminary OD matrix estimate is as follows:

[0017] First, the node feature matrix H (l) is transformed through a learnable weight matrix to generate a new feature representation as: where F (l+1) is the feature dimension of the next layer; for each pair of adjacent nodes i and j in the graph attention network GAT, calculate the corresponding attention coefficient, concatenate or add the features of nodes i and j in H' (l+1) , project them through a learnable weight vector , and apply the non-linear activation function LeakyReLU to calculate the attention score between nodes i and j as follows:

[0018] e(i,j)=LeakyReLU(a (l)T [H' (l+1) (i)||H' (l+1) (j)])

[0019] Then, the attention scores of each node are normalized by the softmax function as follows:

[0020]

[0021] where represents the neighbor set of node i. In the binary adjacency matrix, A(i,k) = 1 only when node i and node k are adjacent. After the preliminary estimation, the final features of all nodes in the graph attention network GAT are represented in matrix form as follows:

[0022] H (l+1) = σ(αH′ (l+1) )

[0023] where represents the attention coefficient matrix of all node pairs, and σ represents the non-linear activation function.

[0024] Furthermore, in step S2, during the time period before time t in the real-time OD matrix for the historical observation data, the historical observation data is denoted as Based on the real-time OD observation data in the historical observation data, the proportion matrix P of various OD pairs in the current time period corresponding to the real-time OD matrix is predicted t , and then the historical OD flow prediction is generated as the historical cumulative result

[0025] Specifically, a time correlation learning model for historical observation data is constructed. This time correlation learning model uses the temporal graph convolutional neural network TGCN to predict the residual distribution, combines the graph neural network GNNs to capture the spatial dependence, and uses the gated recurrent unit GRU to capture the time dependence. In the time correlation learning model, the reset gate r t and the update gate u t are also calculated by the graph convolutional neural network GCN as follows:

[0026] u t = σ(W u [f(A,X t ),h t-1 +b u )

[0027] r t = σ(W r [f(A,X t ),h t-1 +b r )

[0028] c t = tanh(W c [f(A,X t ),(rt *h t-1 )]+b c )

[0029] h t = u t *h t-1 +(1 - u t )*c t

[0030] In the above equations, f(·) represents the graph convolution operation, h t-1 is the hidden state at time t - 1, u t and r t are the update gate and reset gate at time t respectively, h t is the output at time t, W and b are learnable parameters, tanh(·) and σ(·) are activation functions; by stacking multiple temporal graph convolutional neural networks TGCNs, a corresponding prediction module is formed. After taking the historical observation data of the first n time periods of the prediction module and the real-time OD matrix of the current time period as inputs, the output is the historical cumulative result.

[0031] Furthermore, in step S3, the real-time origin flow is denoted as O t , applying the concept of residual learning, based on the pre-interpolation result historical cumulative result and the real-time origin flow O t , allocate the flow of each OD pair in the final OD matrix to be generated; when allocating, satisfy the constraint that the total OD flow of each origin station in the final OD matrix matches the corresponding real-time origin flow O t , that is, achieve the flow conservation constraint.

[0032] Specifically, the process of flow allocation and generating the completed final OD matrix is as follows:

[0033] First, calculate the difference between the pre-interpolation result and the real-time origin flow O t , and represent this difference as a residual, as shown in the following equation:

[0034]

[0035] where the residual R t (i) ∈ R N , represents the unestimated part in the total origin flow at node i, and this residual characterizes the difference between the estimated flow and the actual flow; immediately afterwards, normalize the historical cumulative result to obtain the allocation ratio, and calculate the allocation value corresponding to this allocation ratio using the proportional summation method:

[0036]

[0037] Among them, represents the flow allocation ratio from node i to node j at time t;

[0038] Finally, the calculated residual is allocated to the corresponding positions of the final OD matrix as follows:

[0039]

[0040] Among them, is the final OD matrix In, the final prediction result from node i to node j at time t in the final OD matrix is obtained, thereby realizing the completion of the incomplete real-time OD matrix.

[0041] Preferably, based on the processes of steps S1 to S3, an OD matrix prediction model is constructed; the reconstruction error of the final OD matrix after completion in step S3 is used as the loss function to train the OD matrix prediction model; this loss function is as follows:

[0042]

[0043] Among them, MSE represents the mean square error, represents the final OD matrix, and Y represents the true OD matrix corresponding to the time when the final OD matrix is located; after calculating the loss based on the loss function, the parameters of the OD matrix prediction model are updated using the backpropagation method.

[0044] Specifically, after the construction and parameter update of the OD matrix prediction model are completed, its model inference process is as follows: Obtain the incomplete real-time OD matrix X t and the real-time origin flow O t , input them into the OD matrix prediction model, and generate a preliminary OD matrix estimate after pre-interpolation Obtain historical observation data, input it into the OD matrix prediction model, and generate a historical OD flow prediction after historical accumulation The OD matrix prediction model calculates the residual between the pre-interpolation result the historical OD flow prediction and the real-time origin flow O t , and performs flow allocation to generate the final OD matrix after completion and returns it to the data center. t

[0045] ​​The present invention also provides a real-time OD matrix completion system based on flow conservation constraints. The system uses the GNN-ODFill framework to achieve the completion of the real-time OD matrix. The GNN-ODFill framework includes the following component modules:

[0046] A pre-interpolation module for preliminarily estimating an incomplete real-time OD matrix using a graph attention network (GAT), capturing the spatial dependence between nodes, and generating a preliminary OD matrix estimate after feature transfer as the pre-interpolation result;

[0047] A historical accumulation module for obtaining historical observation data, capturing the spatio-temporal dependence of historical observation data through a combination of a graph convolutional network (GCN) and a gated recurrent unit (GRU), and generating a historical OD flow prediction as the historical accumulation result;

[0048] An OD allocation module for obtaining real-time origin traffic, and performing traffic allocation based on the real-time origin traffic, the pre-interpolation result, and the historical accumulation result through the application of residual learning, and generating a completed final OD matrix under the condition of satisfying the flow conservation constraint.

[0049] In summary, due to the adoption of this technical solution, the method proposed by this solution and the GNN-ODFill framework in the system achieve the following technical effects by combining a graph neural network (GNN) with spatio-temporal dependence modeling and flow conservation constraints:

[0050] 1. Real-time OD matrix completion: GNN-ODFill directly completes an incomplete real-time OD matrix by combining a graph neural network and historical data. By utilizing the complete origin traffic information from each station and combining more complete historical observation data, GNN-ODFill can effectively estimate a complete OD matrix in the short term. This method effectively simplifies the cumbersome compensation process in traditional methods and does not rely on additional auxiliary source data, ensuring real-time performance while improving the completion accuracy.

[0051] 2. Spatio-temporal dependence modeling: The present invention combines a graph convolutional network (GCN) and a gated recurrent unit (GRU) to capture the spatio-temporal dependence of the OD matrix simultaneously. GCN learns spatial dependence through the graph structure, while GRU effectively captures the dynamic pattern of the OD matrix changing over time. This significantly improves the modeling ability of spatio-temporal patterns of the method of the present invention, thereby enhancing the accuracy and stability of OD matrix estimation.

[0052] 3. Effective introduction of flow conservation constraints: The present invention designs a residual processing structure to incorporate flow conservation constraints during the OD matrix completion process, ensuring that the departure flow of each station is consistent with the sum of the arrival flows of each OD pair. This constraint effectively prevents the problem of flow imbalance and improves the rationality and physical consistency of the completion results.

[0053] 4. Improving the accuracy of downstream tasks: The real-time OD matrix completed by using GNN-ODFill can significantly improve the accuracy of downstream tasks (such as traffic flow prediction, traffic demand prediction, etc.). Experiments show that compared with traditional methods, GNN-ODFill can provide more accurate completion results when dealing with real-time missing data, thus providing more reliable data support for traffic flow prediction and management.

[0054] 5. Optimizing computational efficiency: While ensuring high precision, the present invention optimizes computational efficiency and reduces computational complexity. Compared with existing methods, GNN-ODFill reduces the dependence on high-complexity models through a simplified graph neural network architecture, making it suitable for real-time data processing tasks of large-scale traffic networks. Brief Description of the Drawings

[0055] Figure 1 It is a schematic diagram briefly describing the overall process of the method of the present invention;

[0056] Figure 2 It is a schematic diagram of the architecture of the OD matrix prediction model in the present invention;

[0057] Figure 3 It is a schematic diagram of the overall process of the application solution corresponding to the method of the present invention;

[0058] Figure 4 It is an example diagram of real-time OD matrix collection in the application solution of the present invention;

[0059] Figure 5 It is an example diagram of OD matrix generation in the application solution of the present invention;

[0060] Figure 6 It is a schematic diagram of the starting flow change curve of high-flow stations of the subway in Hangzhou and Guangzhou in the data verification experiment of the present invention;

[0061] Figure 7 It is a schematic diagram of the visual comparison of the real-time OD matrix heat map in the data verification experiment of the present invention;

[0062] Figure 8 It is a schematic diagram of the flow change curve of the highest-flow OD pair of the subway in Hangzhou and Guangzhou in the data verification experiment of the present invention;

[0063] Figure 9Schematic diagram of the real-time completion performance of different models as test inputs in the inference mode in the data verification experiment of the present invention. Detailed implementation manners

[0064] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are some, but not all, of the embodiments of the present invention. Components of the embodiments of the present invention described and illustrated herein generally may be arranged and designed in a variety of different configurations.

[0065] Therefore, the following detailed description of the embodiments of the present invention provided in the drawings is not intended to limit the scope of the claimed present invention, but merely represents selected embodiments of the present invention. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0066] Embodiment 1

[0067] A real-time OD matrix completion method for graph neural networks based on flow conservation constraints, Figure 1 For a brief description of the process of this method, its various steps are summarized as follows:

[0068] S1. Pre-imputation: Use a graph attention network (GAT) to perform a preliminary estimation on the incomplete real-time OD matrix, capture the spatial dependence relationship between each node, and generate a preliminary OD matrix estimate after feature transfer as the pre-imputation result;

[0069] S2. Historical accumulation: Obtain historical observation data, and capture the spatio-temporal dependence relationship of the historical observation data by combining a graph convolutional network (GCN) and a gated recurrent unit (GRU) to generate a historical OD flow prediction as the historical accumulation result;

[0070] S3. OD allocation: Obtain the real-time origin traffic, and perform traffic allocation by applying residual learning based on the real-time origin traffic, the pre-imputation result, and the historical accumulation result to generate the final completed OD matrix under the condition of satisfying the flow conservation constraint.

[0071] In this embodiment, under the above method concept, a corresponding OD matrix prediction model can be constructed. The architecture of this model can be seen in Figure 2 , and under its GNN-ODFill framework, corresponding to the method process, it mainly includes three core modules.

[0072] The first is the pre-imputation module (Pre-Imputation), which uses a graph attention network (GAT) on the observed real-time OD matrix X tthe data X in i Message passing is performed on i to generate an initial estimate. The second is the Historical Accumulation module, which combines the Graph Convolutional Network (GCN) and the Gated Recurrent Unit (GRU) to capture the cumulative impact of historical observation data. The third is the OD Allocation module, which integrates the real-time origin flow O t and the residual learning method to allocate OD flows, ensuring that the total amount of the OD estimation result for each site matches the real-time origin flow O corresponding to each site t The following will introduce each step and its module in the method in detail.

[0073] In the pre-imputation and its module of step S1, the principle is to use the Graph Neural Network (GNN) to learn the spatial relationship between different sites and convert the observed incomplete real-time OD matrix into a more complete representation. Specifically, the real-time OD matrix X t represents the observed value with the target time interval t, which is used as the input of the Graph Attention Network (GAT), and the output represents the generated preliminary OD matrix estimate. Among the various spatio-temporal graph neural network architectures that have been proposed, most models rely on weighted adjacency matrices, and these matrices use distance-based Gaussian kernel functions. However, since the open-source OD dataset usually only indicates whether nodes are adjacent and does not provide distance information, in this embodiment, the Graph Attention Network (GAT) is constructed only by building a binary adjacency matrix; the Graph Attention Network (GAT) is one of the graph neural networks (GNNs), and it is also one of the core components of the architecture selected in this embodiment to ensure the message passing weights between nodes.

[0074] In the Graph Attention Network (GAT) of this embodiment, the node feature matrix where N is the number of nodes, l is the number of layers, and F (l) is the feature dimension of the l-th layer, represents the set of real numbers; the real-time OD matrix X t is used as the initial feature matrix H (0) , that is, H (0) = X t ; the process of generating the preliminary OD matrix estimate through preliminary estimation is as follows:

[0075] First, the node feature matrix H (l) is transformed through a learnable weight matrix to generate a new feature representation as: where F (l+1)is the feature dimension of the next layer; for each pair of adjacent nodes i and j in the graph attention network GAT, calculate the corresponding attention coefficient, concatenate or add the features of nodes i and j in H′ (l+1) and project them through a learnable weight vector and apply the non-linear activation function LeakyReLU to calculate the attention score between nodes i and j, as shown in the following formula:

[0076] e(i,j) = LeakyReLU(a (l)T [H′ (l+1) (i)||H′ (l+1) (j)])

[0077] Then, normalize the attention scores of each node through the softmax function, as shown in the following formula:

[0078]

[0079] where represents the neighbor set of node i, indicating that in the binary adjacency matrix, only when node i and node k are adjacent, A(i,k) = 1, which ensures that information does not aggregate from non-adjacent nodes; after preliminary estimation, the final features of all nodes in the graph attention network GAT are represented in matrix form as follows:

[0080] H (l+1) = σ(αH′ (l+1) )

[0081] where represents the attention coefficient matrix of all node pairs, and σ represents the non-linear activation function.

[0082] In the historical accumulation and its module of step S2, in the time period before time t, there will also be a relatively complete set of historical observation data, which is more complete than the current real-time OD matrix X t . In this embodiment, these historical observation data will help improve the estimation of the real-time OD data distribution because OD data has temporal continuity and correlation patterns. This continuity indicates that the patterns of OD flows usually persist within short time intervals, and historical observation data can capture typical fluctuations and trends, and this information can be used to enhance the accuracy and stability of real-time OD estimation.

[0083] Specifically in this embodiment, the historical observation data is denoted as Based on the real-time OD observation data in the historical observation data, predict the proportion matrix P t of various OD pairs in the current time period corresponding to the real-time OD matrix, and then generate a historical OD flow prediction as the historical accumulation result Meanwhile, a time correlation learning model for historical observation data is constructed. The time correlation learning model uses the temporal graph convolutional neural network TGCN to predict the residual distribution, combines the graph neural network GNNs to capture spatial dependencies, and uses the gated recurrent unit GRU to capture temporal dependencies. In the time correlation learning model, the reset gate r is also calculated through the graph convolutional neural network GCN t and the update gate u t , as follows:

[0084] u t = σ(W u [f(A, X t ), h t-1 + n u )

[0085] r t = σ(W r [f(A, X t ), h t-1 + b r )

[0086] c t = tanh(W c [f(A, X t ), (r t * h t-1 )] + b c )

[0087] h t = u t * h t-1 + (1 - u t ) * c t

[0088] In the above formulas, f(·) represents the graph convolution operation, h t-1 is the hidden state at time t - 1, u t and r t are the update gate and reset gate at time t respectively, h t is the output at time t, W and b are learnable parameters, tanh(·) and σ(·) are activation functions; by stacking multiple temporal graph convolutional neural networks TGCNs, a corresponding prediction module is formed. After taking the historical observation data of the first n time periods of the prediction module and the real-time OD matrix of the current time period as inputs, the output is the historical cumulative result.

[0089] In the OD allocation and its module of step S3, in this embodiment, the concept of residual learning is applied, and based on the pre-interpolation result historical cumulative result and the real-time origin flow O t, allocate the flow of each OD pair in the final OD matrix to be generated; when allocating, satisfy the constraint that the total OD flow of each origin station in the final OD matrix matches the corresponding real-time origin flow O t to achieve the flow conservation constraint.

[0090] In this embodiment, the process of performing flow allocation and generating the completed final OD matrix is as follows:

[0091] First, calculate the pre-interpolation result and the difference between it and the real-time origin flow O t , and represent this difference as a residual, as shown in the following formula:

[0092]

[0093] where the residual R t (i) ∈ R N , represents the unestimated part in the total origin flow at node i, and this residual characterizes the difference between the estimated flow and the actual flow. These residual information will be used to fill the relevant gaps in the subsequent process.

[0094] Immediately, a distribution ratio is needed to normalize the historical cumulative result to obtain this distribution ratio; the reason for calculating this ratio through historical observation data is that historical observation data is usually more complete than real-time data and can reflect a relatively stable real-time OD distribution. Different from using sigmoid or other non-linear activation functions for normalization in this embodiment, the ratio summation method is adopted to calculate the distribution value corresponding to this distribution ratio, as shown in the following formula:

[0095]

[0096] where, represents the flow distribution ratio from node i to node j at time t;

[0097] Finally, allocate the calculated residual to the corresponding position in the final OD matrix , as shown in the following formula:

[0098]

[0099] where, is the final prediction result from node i to node j at time t in the final OD matrix , thus completing the incomplete real-time OD matrix.

[0100] In this embodiment, the OD matrix obtained from real-time observations is a special case of missing data. In this case, it is impossible to accurately identify which parts of the OD flow are not fully observed. Due to the randomness of traveler behavior, even short-distance OD flows may lack complete observation data, making it difficult to determine the specific missing entries. To account for this uncertainty and improve the robustness of the OD matrix prediction model in various scenarios, this embodiment uses the reconstruction error of the final OD matrix after completion as the loss function to train the OD matrix prediction model; this loss function is as follows:

[0101]

[0102] where MSE represents the mean square error, represents the final OD matrix, and Y represents the true OD matrix corresponding to the time when the final OD matrix is located; after calculating the loss based on the loss function, the parameters of the OD matrix prediction model are updated using the backpropagation method.

[0103] Embodiment 2

[0104] Based on Embodiment 1, this embodiment introduces the overall application scheme of the method in detail and makes an exemplary demonstration. When applying, this embodiment is based on the collaborative work of a deep learning model and a traffic sensor network real-time monitoring system. Through the real-time monitoring of system data and the generation of the OD matrix and origin departure flow, and the real-time information transfer between the server-side model, the real-time monitoring of data and the completion of the OD matrix are jointly completed; the overall process of the application scheme can be seen in Figure 3 the schematic diagram of.

[0105] This embodiment specifically divides this process into the data side and the model side and introduces them separately. First is the introduction of the data side, as follows:

[0106] Principle of real-time OD missing: Figure 4 Shows an example of collecting a real-time OD matrix at 15-minute intervals. Suppose two passengers depart from station 0, increasing the departure flow of station 0 by 2. In the next 15 minutes, only one passenger arrives at station 1, while the other passenger is still on the way. Therefore, the OD flow between station 0 and station 1 increases by 1, but the flow between station 0 and station 2 remains unchanged. Although in reality, the OD flow from station 0 to station 2 actually increases by 1, due to the limitation of the 15-minute real-time observation window, this increase cannot be recorded.

[0107] Principle of data generation: Figure 5 Respectively show the adjacency matrix A, the true value matrix Y, and the real-time observation matrix X of four stations. As Figure 5As shown, when the origin traffic of node 0 is 100, the corresponding traffic from node 0 to nodes 1, 2, and 3 should be 41, 32, and 27 respectively, and these traffic are represented in Figure 5 the true value matrix Y in (b). However, due to travel and time limitations, only 31 people arrived at node 1 during the observation period, and the remaining traffic is still on the way. Therefore, Figure 5 the real-time observation matrix X shown in (c) reflects this incomplete traffic distribution.

[0108] In this embodiment, in order to ensure the training of the OD matrix prediction model, a reliable training data set needs to be generated from a large amount of historical observation data. Therefore, a real-time incomplete OD matrix needs to be simulated from the historical observation data, that is, within a limited time interval, according to the foregoing generation principle, the corresponding X and Y are generated. In terms of the adjacency matrix structure, the simplest binary adjacency matrix is adopted and constructed according to the definition of the undirected graph, as shown in the following formula:

[0109]

[0110] The above application process of the corresponding data end can fully use the method process of Embodiment 1. The following is a complete introduction of the corresponding model end.

[0111] In the training process of the OD matrix prediction model, the input of the pre-imputation module is the incomplete real-time OD matrix X t , after using the graph attention network GAT for feature transfer, a preliminary OD matrix estimate is generated The input of the historical accumulation module is a large amount of historical data X i , using the graph convolutional network GCN and the gated recurrent unit GRU to capture the spatio-temporal dependence relationship, and generating the historical OD traffic prediction The input of the OD allocation module is the pre-imputation result the historical accumulation result and the real-time origin traffic O t , by calculating the residual R t , performing traffic allocation to ensure traffic conservation, and generating the final OD matrix Subsequently, the mean square error MSE is used to calculate the loss, and the parameters of the OD matrix prediction model are updated by the backpropagation method.

[0112] After the construction and parameter update of the OD matrix prediction model are completed, its model inference process is as follows: Obtain the incomplete real-time OD matrix X t and the real-time origin traffic O t in the actual application, input them into the OD matrix prediction model, and generate a preliminary OD matrix estimate after pre-imputation Obtain the historical observation data, input it into the OD matrix prediction model, and generate the historical OD traffic prediction after historical accumulation The OD matrix prediction model estimates based on the preliminary OD matrix Predict historical OD flows and the real-time origin flow O t , calculate the pre-imputation result and the real-time origin flow O t Calculate the residual between them, perform flow allocation, and generate the final completed OD matrix and return it to the data center

[0113] Example 3

[0114] Based on any of the above embodiments, this embodiment correspondingly introduces the verification of the effectiveness of the OD matrix prediction model in estimating real-time OD flows. This embodiment conducts experiments on three traffic datasets: Hangzhou Metro, Guangzhou Metro, and Manhattan taxis. The relevant links are as follows:

[0115] https: / / www.kaggle.com / datasets / zjplab / hangzhou-metro-traffic-prediction

[0116] https: / / www.nyc.gov / site / tlc / businesses / yellow-cab.page

[0117] Each traffic dataset represents different urban travel patterns and infrastructure characteristics, enabling this embodiment to evaluate the adaptability of the OD matrix prediction model in different scenarios. This embodiment compares the performance of the OD matrix prediction model with the current optimal existing methods on each traffic dataset

[0118] The method compared with the model of this embodiment is the current optimal OD prediction method HIAM. This embodiment also evaluates the OD matrix prediction model with other existing models that can solve related problems. Currently, some existing methods do not directly address the problem of real-time OD estimation. However, their models can be adapted to the framework of the OD matrix prediction model of this embodiment by taking incomplete historical OD as input and generating the complete OD for the current time period as output

[0119] Table 1 below shows that the OD matrix prediction model (identified as GNN-ODFill) of this embodiment always outperforms other models in all datasets and evaluation metrics (MSE, RMSE, MAE, MAPE). This can be attributed to multiple key features in the model of this embodiment

[0120]

[0121] To further verify the importance of introducing the traffic conservation constraint, Figure 6 shows the variation of the origin traffic throughout the day at representative high-traffic stations in the Hangzhou Metro and Guangzhou Metro datasets. The origin traffic estimates for all models are obtained by summing the predicted OD flows. As Figure 6 (a) shows, the origin traffic prediction of the model in this embodiment is highly consistent with the true value of the Hangzhou dataset. Similarly, in Figure 6 (b), the model in this embodiment is the only one that accurately fits the actual origin traffic of the Guangzhou dataset, and the minor deviation in the final model output is mainly due to rounding. In contrast, other models fail to achieve this accuracy, especially on the Guangzhou dataset. Without incorporating the origin traffic information, the estimated OD matrix may violate physical constraints, resulting in unrealistic results and thus having a negative impact on downstream tasks.

[0122] To visually demonstrate the effectiveness of the OD matrix prediction model in this embodiment, the OD matrix completion performance of different models is compared during the peak hours of three datasets here. The daily peak hours of each dataset are determined, and the completion results are shown in Figure 7 . To make the comparison clearer and more intuitive, Figure 7 only the top 50 stations with the highest origin traffic are shown separately. The observation results show that there are significant differences between the real-time observed OD matrix (Observed) and the true OD matrix (Truth). However, the OD matrix prediction model in this embodiment successfully completes the OD matrix to a level close to the true value. The HIAM model obtains results comparable to those of the model in this embodiment on the Hangzhou Metro and Manhattan taxi datasets, but on the Guangzhou Metro dataset, the model in this embodiment is significantly better than HIAM.

[0123] Figure 8 shows the traffic variation curves of the highest-traffic OD pairs throughout the day in the Hangzhou Metro and Guangzhou Metro.

[0124] These curves include real-time observations, true values, and the predicted values of the model in this embodiment, HA, HIAM, and STID models. As Figure 8 (a) shows, during off-peak hours, the fitting effects of all models are quite good, but only the model in this embodiment can accurately fit the traffic during peak hours. HIAM follows closely, and although the real-time observations of some OD pairs are zero, the output of the model in this embodiment is still highly consistent with the true value, demonstrating the effectiveness of the GNN component in the model in this embodiment. In Figure 8 (b), although the true values show more fluctuations, the model in this embodiment still provides a relatively good fit, indicating its higher accuracy in real-time completion.

[0125] This embodiment also tested the improvement effect of the data after different models were completed on their respective corresponding prediction models, which can be seen in Figure 9 the real-time completion performance schematic diagram under the shown inference mode, where the green left axis of each model in each subgraph represents RMSE, and the red right axis represents MAPE. From Figure 9 the figure, it can be seen that the completion result of the OD matrix prediction model in this embodiment has a relatively ideal improvement on OD prediction.

Claims

1. A graph neural network real-time OD matrix completion method based on flow conservation constraints, characterized in that: The method comprises the following steps: S1. Pre-interpolation: Use the graph attention network GAT to make a preliminary estimate of the incomplete real-time OD matrix, capture the spatial dependencies between nodes, and generate a preliminary OD matrix estimate after feature transfer as the pre-interpolation result; S2. Historical accumulation: Obtain historical observation data, capture the spatiotemporal dependencies of historical observation data by combining graph convolutional networks (GCN) and gated recurrent units (GRU), and generate historical OD traffic predictions as historical accumulation results; S3, OD allocation: obtain the real-time originating flow, and allocate the flow by applying residual learning based on the real-time originating flow, pre-interpolation results and historical accumulation results, and generate the final OD matrix after completion under the condition of satisfying the flow conservation constraint.

2. The graph neural network real-time OD matrix completion method according to claim 1, characterized in that: In step S1, the incomplete real-time OD matrix is ​​recorded as X t , represents the observation with the target time interval t, which is used as the input of the graph attention network GAT; the output of the graph attention network GAT is recorded as Represents the generated preliminary OD matrix estimate; the construction of the graph attention network GAT is completed by constructing a binary adjacency matrix.

3. The graph neural network real-time OD matrix completion method according to claim 2, characterized in that: In the graph attention network GAT, the node feature matrix Where N is the number of nodes, l is the number of layers, and F (l) is the feature dimension of the lth layer, represents a real number set; the real OD matrix X t As the initial feature matrix H (0) , that is, H (0) =X t ; The process of making preliminary estimates to generate preliminary OD matrix estimates is as follows: First, the node feature matrix H (l) Through the learnable weight matrix Transform and generate new features represented as: where F (l+1) is the feature dimension of the next layer; For each pair of adjacent nodes i and j in the graph attention network GAT, the corresponding attention coefficient is calculated and nodes i and j are placed in H′ (l+1) The features in are concatenated or added together through a learnable weight vector Projection is performed and the nonlinear activation function LeakyReLU is applied to calculate the attention score between nodes i and j as follows: e(i,j)=LeakyReLU(a (l)T [H′ (l+1) (i)||H′ (l+1) (j)]) Then, the attention score of each node is normalized by the softmax function as follows: in Represents the neighbor set of node i, which means that in the binary adjacency matrix, A(i,k) = 1 only when node i and node k are adjacent. After preliminary estimation, the final features of all nodes in the graph attention network GAT are expressed in matrix form as follows: H (l+1) =σ(αH′ (l+1) ) in represents the attention coefficient matrix of all node pairs, and σ represents the nonlinear activation function.

4. The graph neural network real-time OD matrix completion method according to claim 2, characterized in that: In step S2, the historical observation data is in the time period before time t of the real-time OD matrix, and the historical observation data is recorded as According to the real-time OD observation data in the historical observation data, the proportion matrix P of various OD pairs in the current time period corresponding to the real-time OD matrix is ​​predicted t , and then generate historical OD traffic forecasts as historical accumulation results 5. The graph neural network real-time OD matrix completion method according to claim 4, characterized in that: A time correlation learning model for historical observation data is constructed. The time correlation learning model uses a time series graph convolutional neural network (TGCN) to predict the residual distribution, combines graph neural networks (GNNs) to capture spatial dependencies, and uses gated recurrent units (GRUs) to capture temporal dependencies. In the time correlation learning model, the reset gate r is also calculated through the graph convolutional neural network (GCN). t and update gate u t ,as follows: u t =σ(W u [f(A,X t ),h t-1 ]+b u ) r t =σ(W r [f(A,X t ),h t-1 ]+b r ) c t =tanh(W c [f(A,X t ),(r t *h t-1 )]+b c ) h t =u t *h t-1 +(1-u t )*c t In the above formulas, f(·) represents the graph convolution operation, h t-1 is the hidden state at time t-1, u t and r t are the update gate and reset gate at time t, respectively, and h t is the output at time t, W and b are learnable parameters, tanh(·) and σ(·) are activation functions; by stacking multiple time series graph convolutional neural networks TGCN, a corresponding prediction module is formed, and the historical observation data of the previous n time periods and the real-time OD matrix of the current time period are taken as input to output This is the historical cumulative result.

6. The graph neural network real-time OD matrix completion method according to claim 4, characterized in that: In step S3, the real-time originating flow is recorded as O t , applying the residual learning concept, based on the pre-interpolation results Historical cumulative results and real-time origination flow O t , distribute the flow of each OD pair in the final OD matrix to be generated; when distributing, the total OD flow of each starting station in the final OD matrix is ​​satisfied with the corresponding real-time originating flow O t Matching constraints, that is, realizing flow conservation constraints.

7. The graph neural network real-time OD matrix completion method according to claim 6, characterized in that: The process of performing traffic distribution and generating the final OD matrix after completion is as follows: First, calculate the pre-interpolation results With real-time origination flow O t The difference between them is expressed as the residual, as follows: Among them, the residual R t (i)∈R N , represents the unestimated part of the total originating flow at node i. The residual represents the difference between the estimated flow and the actual flow. Next, the historical cumulative results are Normalize to get the allocation ratio, and use the ratio summation method to calculate the allocation value corresponding to the allocation ratio: in, represents the traffic distribution ratio from node i to node j at time t; Finally, the calculated residual is assigned to the final OD matrix The corresponding position is as follows: in, The final OD matrix is In , the final prediction result from node i to node j at time t is obtained, thereby completing the incomplete real-time OD matrix.

8. The graph neural network real-time OD matrix completion method according to claim 1, characterized in that: Based on the process from step S1 to S3, an OD matrix prediction model is constructed; the reconstruction error of the final OD matrix completed in step S3 is used as a loss function to train the OD matrix prediction model; the loss function As follows: Among them, MSE represents mean square error, represents the final OD matrix, Y represents the real OD matrix corresponding to the moment of the final OD matrix; after the loss is calculated according to the loss function, the back propagation method is used to update the parameters of the OD matrix prediction model.

9. The graph neural network real-time OD matrix completion method according to claim 8, characterized in that: After the construction and parameter update of the OD matrix prediction model is completed, the model reasoning process is as follows: Obtain the incomplete real-time OD matrix X in actual application t and real-time origination flow O t , input into the OD matrix prediction model, and generate the preliminary OD matrix estimate after pre-interpolation Obtain historical observation data, input it into the OD matrix prediction model, and generate historical OD flow prediction after historical accumulation The OD matrix prediction model is based on the preliminary OD matrix estimation Historical OD traffic forecast and real-time origination flow O t , calculate the pre-interpolation result With real-time origination flow O t The residual between them is used to distribute the traffic and generate the final OD matrix after completion. And return to the data center.

10. A graph neural network real-time OD matrix completion system based on flow conservation constraints, characterized in that: The system adopts the GNN-ODFill framework to complete the real-time OD matrix. The GNN-ODFill framework includes the following components: The pre-interpolation module is used to use the graph attention network GAT to make a preliminary estimate of the incomplete real-time OD matrix, capture the spatial dependencies between nodes, and generate a preliminary OD matrix estimate as the pre-interpolation result after feature transfer; The historical accumulation module is used to obtain historical observation data. By combining the graph convolutional network (GCN) and the gated recurrent unit (GRU), it captures the spatiotemporal dependencies of historical observation data and generates historical OD traffic predictions as historical accumulation results. The OD allocation module is used to obtain the real-time originating flow, and to allocate the flow by applying residual learning based on the real-time originating flow, pre-interpolation results and historical accumulation results, and to generate the final OD matrix after completion under the condition of satisfying the flow conservation constraint.

Images