Traffic state prediction method based on multi-modal data and adaptive topology modeling

Abstract

The embodiment of the invention discloses a traffic state prediction method based on multi-modal data and adaptive topology modeling. The method comprises the following steps: dynamically determining an adjacent matrix between roads according to the type and historical traffic flow of each road, and respectively inputting a GCN model and a GAT model according to an adjacent matrix dynamic traffic network diagram; extracting global spatial features by the GCN model according to the weight of each edge and the current feature representation of each node; strengthening the effect of a key node by the GAT model to obtain a local attention space feature; performing short-time timing sequence modeling and long-time timing sequence modeling on the time sequence of the multi-modal traffic data of each node in the latest period of time to obtain a short-time feature and a long-time feature respectively; and predicting a future traffic state according to the global spatial features, the local attention spatial features, the short-time features and the long-time features. According to the embodiment, the traffic state prediction accuracy is improved.

Description

Technical Field

[0001] The embodiments of the present invention relate to the technical fields of intelligent transportation and artificial intelligence, and in particular to a traffic state prediction method based on multi-modal data and adaptive topology modeling. Background Art

[0002] With the development of intelligent transportation systems, urban traffic prediction technology has become one of the key technologies for improving road traffic efficiency and optimizing traffic management strategies. Currently, the main traffic prediction methods can be classified into the following categories:

[0003] 1. Methods based on statistical modeling: such as time series analysis methods like ARIMA (Auto Regressive Integrated Moving Average) and SARIMA (Seasonal Auto Regressive Integrated Moving average), which perform predictions through historical traffic flow data. However, their linear assumptions limit the modeling ability for complex non-linear traffic flows and it is difficult to capture the spatial dependence relationships between roads.

[0004] 2. Graph neural networks based on fixed adjacency matrices: Use methods such as graph neural networks for spatial modeling, but use static adjacency matrices and cannot dynamically adapt to different road types and real-time traffic flow changes, resulting in limited prediction accuracy.

[0005] 3. Traffic prediction based on temporal deep learning: Only models traffic changes in time and fails to effectively consider the spatial interactions between roads, making it difficult to accurately predict traffic anomalies.

[0006] Patent application CN119274345A provides a spatio-temporal correlation traffic flow prediction method based on deep learning, and patent application CN116071923A provides a traffic flow prediction method based on an adaptive graph fusion convolutional network, which also cannot well solve the above problems. Summary of the Invention

[0007] The embodiments of the present invention provide a traffic state prediction method based on multi-modal data and adaptive topology modeling to solve at least one of the above technical problems.

[0008] In a first aspect, the embodiments of the present invention provide a traffic state prediction method based on multi-modal data and adaptive topology modeling, including:

[0009] Obtain an initial traffic network graph of the area to be predicted, where the traffic network graph takes roads as nodes and the connection relationships between roads as edges;

[0010] Dynamically determine the adjacency matrix between roads according to the types of each road and the historical traffic flow of each road in a recent period of time, where each element in the adjacency matrix is used to characterize the degree of mutual influence of traffic between two roads;

[0011] In the initial traffic network graph, assign weights to each edge according to the adjacency matrix, and construct the current feature representation of each node according to the current multi-modal traffic data of each road to obtain a dynamic traffic network graph;

[0012] Input the dynamic traffic network graph into the GCN model and the GAT model respectively; the GCN model extracts global spatial features according to the weights of each edge and the current feature representation of each node; the GAT model strengthens the role of key nodes to obtain local attention spatial features;

[0013] Perform short-term time series modeling and long-term time series modeling on the time series of each node's multi-modal traffic data in a recent period of time to obtain short-term features and long-term features respectively;

[0014] Predict the future traffic state according to the global spatial features, local attention spatial features, short-term features and long-term features.

[0015] In a second aspect, an embodiment of the present invention provides an electronic device, and the electronic device includes:

[0016] One or more processors;

[0017] A memory for storing one or more programs,

[0018] When the one or more programs are executed by the one or more processors, the one or more processors implement the traffic state prediction method based on multi-modal data and adaptive topology modeling according to any embodiment.

[0019] In a third aspect, an embodiment of the present invention further provides a computer-readable storage medium, on which a computer program is stored, and when the program is executed by a processor, it implements the traffic state prediction method based on multi-modal data and adaptive topology modeling according to any embodiment.

[0020] In summary, the embodiment of the present invention provides a traffic state prediction method based on multi-modal data and adaptive topology modeling, which fuses multi-modal traffic data and optimizes data quality to provide a good data foundation for traffic state prediction. In traffic state prediction, based on road categories and historical traffic flows, this embodiment constructs an adjacency matrix with adaptive changes to dynamically adjust the traffic network topology, enabling it to adjust the influence weights between roads over time. Subsequently, it is applied to a spatial feature extraction model, where the global topology information of the road network is dynamically extracted through a GCN model, and the dependence relationship between key roads is strengthened using the multi-layer attention mechanism of a GAT model. At the same time, the optimized multi-modal data time series are respectively input into LSTM and Transformer models to model short-term emergencies and long-term travel patterns, further improving the prediction accuracy. Finally, the global spatial features, local attention spatial features, short-term time series features, and long-term time series features are fused and input into a multi-layer perceptron for regression calculation to output future traffic flow, vehicle speed, and congestion index. Through adaptive dynamic topology modeling, graph neural network optimization, and time series feature fusion, this embodiment effectively improves the accuracy and robustness of traffic state prediction and can be applied to intelligent traffic management, signal optimization, road planning, and congestion governance to provide more efficient data support. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] In order to more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the following will briefly introduce the drawings required for use in the description of the specific embodiments or the prior art. Obviously, the following drawings are some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0022] Figure 1 It is a flowchart of a traffic state prediction method based on multi-modal data and adaptive topology modeling provided by an embodiment of the present invention;

[0023] Figure 2 It is a flowchart of another traffic state prediction method based on multi-modal data and adaptive topology modeling provided by an embodiment of the present invention;

[0024] Figure 3 It is a schematic structural diagram of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0025] To make the objectives, technical solutions and advantages of the present invention clearer, the technical solutions of the present invention will be described clearly and completely below. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all of the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the scope of protection of the present invention.

[0026] In the description of the present invention, it should be noted that the orientation or positional relationship indicated by the terms "center", "upper", "lower", "left", "right", "vertical", "horizontal", "inner", "outer", etc. is based on the orientation or positional relationship shown in the drawings. It is only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and thus cannot be construed as a limitation of the present invention. In addition, the terms "first", "second", "third" are only used for descriptive purposes and cannot be construed as indicating or implying relative importance.

[0027] In the description of the present invention, it should also be noted that unless otherwise clearly defined and limited, the terms "installed", "connected", "connected" should be understood in a broad sense. For example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be directly connected, or indirectly connected through an intermediate medium, and it can be the communication inside two elements. For those of ordinary skill in the art, the specific meanings of the above terms in the present invention can be understood according to specific situations.

[0028] Figure 1 is a flowchart of a traffic state prediction method based on multi-modal data and adaptive topology modeling provided by an embodiment of the present invention. This method realizes accurate urban traffic state prediction through multi-modal data fusion and adaptive traffic network topology, and can be widely applied to application scenarios such as urban traffic flow prediction, signal light optimization, road planning, and congestion management. This method is executed by an electronic device, such as Figure 1 shown, and specifically includes:

[0029] S110. For the area to be predicted, collect the GPS data of operating vehicles, mobile phone signaling data, fixed monitoring point data, meteorological data, and traffic event data in this area.

[0030] In this embodiment, multi-modal data collection and processing related to the traffic state of the urban road network are first carried out, aiming to provide a high-quality data basis for subsequent traffic prediction models.

[0031] Figure 2It is a flowchart of another traffic state prediction method based on multi-modal data and adaptive topology modeling provided by an embodiment of the present invention, showing the key links of the method in this embodiment from another perspective. Specifically, in combination with Figure 1 and Figure 2 , for the future moment to be predicted in this embodiment, first, multi-modal data collection and processing are performed on a period of time ending at this moment. The data sources may include:

[0032] (1) GPS (Global Positioning System) data of operating vehicles

[0033] Data source: mainly from long-distance passenger transport, taxis, buses, online car-hailing, etc. equipped with GPS devices.

[0034] Sampling frequency: usually 1 second to 30 seconds (adjustable).

[0035] Main fields:

[0036]

[0037] Among them:

[0038] represents the GPS data of the vehicle at time ;

[0039] and respectively represent the longitude and latitude of the vehicle at time ;

[0040] represents the speed of the vehicle at time (unit: km / h, kilometers per hour);

[0041] represents the vehicle at time ;

[0042] represents the vehicle at time (such as passenger-carrying / vacant, running / stopped).

[0043] The GPS data of operating vehicles is high-frequency data, which can reflect the changes in microscopic traffic flow on the road, such as queuing, traffic light start-up, etc. The GPS data of operating vehicles can be used for trajectory inference, calculation of local road traffic flow, auxiliary accident detection and signal optimization, etc.

[0044] (2) Mobile phone signaling data

[0045] Data source: anonymized mobile phone base station location information provided by mobile carriers.

[0046] Update frequency: Sampling once every 5 - 10 minutes (depending on the user's movement status and base station handover frequency).

[0047] Main fields:

[0048]

[0049] Among them:

[0050] Represents the mobile phone signaling data of a mobile phone user At time ;

[0051] Represents the base station ID connected by a mobile phone user At time ;

[0052] Records the signaling timestamp.

[0053] Mobile phone signaling data has a wide coverage and can supplement the travel information of private vehicles and pedestrians that cannot be obtained by GPS data. Mobile phone signaling data can be used for OD (Origin Destination) analysis, calculating the origin-destination distribution, and identifying the crowd tidal effect.

[0054] (3) Traffic flow data of fixed monitoring points

[0055] Data source: Fixed monitoring devices such as inductive loops, microwave radars, and high-definition cameras on highways and urban arterial roads.

[0056] Sampling frequency: Updated once every 1 - 5 minutes (can be set by the device).

[0057] Main fields:

[0058] : Represents the traffic flow of a road At time (unit: vehicles / hour);

[0059] : Represents the average vehicle speed of a road At time (unit: km / h).

[0060] The traffic flow data of fixed monitoring points provides long-term stable data and can be combined with mobile data for calibration. The traffic flow data of fixed monitoring points can be used to supplement the possible sampling biases of individual behavior data (GPS data of operating vehicles, mobile phone signaling data).

[0061] (4)Meteorological data

[0062] Data source: API (Application Programming Interface) of the National Meteorological Administration, commercial meteorological data services.

[0063] Sampling frequency: updated hourly.

[0064] Main fields:

[0065]

[0066] Among them:

[0067] Indicates the moment of meteorological data

[0068] Indicates the moment of rainfall (mm);

[0069] Indicates the moment of temperature (°C);

[0070] Indicates the moment of visibility (m).

[0071] Traffic is greatly affected by weather. Rainy / snowy days will reduce vehicle speed and increase accident rate. Combining meteorological data with other data can analyze the impact of weather on the road network and improve prediction accuracy.

[0072] (5)Traffic incident data

[0073] Data source: accident reporting system of traffic management departments, real-time event reports on social media.

[0074] Sampling frequency: updated in real time.

[0075] Main fields:

[0076]

[0077] Among them:

[0078] Indicates the moment of traffic incident data;

[0079] Indicates the moment of traffic incident type (accident, construction, road closure, etc.);

[0080] Indicates the moment Location of traffic incident;

[0081] Indicating a moment Severity of traffic (0 - 10).

[0082] Traffic incident data can predict abnormal congestion, such as a decrease in traffic flow in a construction area or queues induced by accidents. Traffic incident data can be used for causal reasoning to evaluate the impact of incidents on traffic.

[0083] As described above, this embodiment takes into account high-precision trajectories (GPS), macroscopic travel patterns (signaling), and long-term monitoring (fixed monitoring points), provides comprehensive congestion causes (weather + incidents), and lays a data foundation for subsequent modeling. After data collection, through time alignment, deduplication, and fusion, an original multi-modal data set is formed :

[0084]

[0085] Among them, and respectively represent , , , , , , and respectively, after taking the union for all vehicles, mobile phone users, roads, and / or moments, the sets obtained respectively.

[0086] Then, the original multi-modal data set is cleaned, noise-reduced, and scene-classified to improve data quality and ensure the accuracy and robustness of subsequent modeling. In a specific embodiment, this process may include the following steps:

[0087] Step 1: Detect and process outliers: Remove sensor errors and extreme outliers from the data. Optionally, the sources of data anomalies include: equipment errors, such as GPS and sensors may generate incorrect values due to malfunctions, such as unreasonable speeds (>300 km / h); environmental interference, such as mobile phone signaling data may be affected by base station handovers and cause position drift; and human factors, such as incorrect data input or loss. Optionally, the outlier detection methods include:

[0088] (1) Z-score method

[0089] This method is applicable to numerical data (such as traffic flow and vehicle speed), and calculates the standardized score of the data :

[0090]

[0091] Among them, represents the data value of the current feature, represents the mean value of the feature, represents the standard deviation of the feature. If ∣Z∣>3, it is considered an outlier (a point outside 3 times the standard deviation).

[0092] (2) Box plot method

[0093] This method is applicable to data such as flow rate, speed, precipitation, etc., and calculates the interquartile range :

[0094]

[0095] Among them, represents the 25th percentile, represents the 75th percentile. If the data value , or , then it is considered that is an outlier.

[0096] (3) Speed jump detection

[0097] This method is applicable to GPS data and calculates the speed change rate of two consecutive GPS points of the same vehicle :

[0098]

[0099] Among them, and respectively represent the speeds of the same vehicle extracted from GPS data at two adjacent moments. If a threshold is set (such as 50 km / h / s), it is regarded as abnormal data.

[0100] (4) Abnormal detection of signaling data

[0101] If a user travels more than 100 kilometers within 5 minutes, then this signaling point may be affected by base station handover and needs to be excluded.

[0102] Step 2. Perform missing value filling: Ensure the integrity of the data and prevent the model from making misjudgments due to missing data. Missing data may lead to misjudgments of the prediction model. In this embodiment, different methods are used to fill different types of missing data, as shown in Table 1:

[0103] Table 1

[0104] Data type Filling method Applicable scenario Traffic flow #timg# Linear interpolation Short-term traffic flow loss Vehicle speed #timg# Linear interpolation Speed fluctuation between measurement points GPS trajectory Bézier curve interpolation Trajectory breakpoint filling Mobile phone signaling Nearest neighbor interpolation 5 - 10 minute gap filling Meteorological data Historical mean replacement Short-term loss of meteorological data

[0105] Specifically, (1) Linear interpolation: Applicable to short-term missing of time series:

[0106]

[0107] Among them, represents the data value at the missing moment, and respectively represent the data values at the two moments before and after the missing moment.

[0108] (2) B-spline curve interpolation: applicable to GPS trajectories:

[0109]

[0110] Among them, represents the missing data point in the trajectory, , and respectively represent the reference point before the missing point, the estimated control point at the missing point, and the reference point after the missing point in the trajectory, represents the normalized time progress parameter (usually obtained by linear time mapping, with a range of [0, 1]).

[0111] (3) Nearest neighbor interpolation: applicable to mobile phone signaling:

[0112]

[0113] Among them, represents the data value at the missing moment, represents the data value at the moment before the missing moment.

[0114] Step 3: Perform data denoising: reduce measurement errors and random jitters, and extract the true traffic trend. Optional data denoising methods include:

[0115] (1) Moving average method: applicable to traffic flow and vehicle speed data.

[0116]

[0117] Among them, represents the original data, represents the smoothed data, represents the moment as the center of the moving time window length. When, short-term fluctuations can be smoothed.

[0118] (2) Kalman filter: applicable to GPS trajectories.

[0119]

[0120] Among them, and respectively represent the data values at two adjacent moments after filtering, and respectively represent the state transition matrix in the Kalman filter, represents Gaussian noise, represents the control input quantity, such as the influence of external actions such as the acceleration or direction change of the vehicle.

[0121] It should be noted that in the above several data preprocessing methods, some different methods use the same variables. In this case, the meanings of each variable should be treated differently according to the specific description of the variable in the method.

[0122] Step 4: Perform scenario classification: Reasonably group the data according to road characteristics and traffic patterns to support the adaptive prediction model. Optionally, the scenario classification in this embodiment can be understood as road type classification. Since the traffic rules of different road types are different, the following scenarios can be divided:

[0123] Urban road: Marked as with traffic rules of many traffic lights and low vehicle speed;

[0124] Suburban road: Marked as with traffic rules of few traffic lights and relatively stable speed;

[0125] Highway: Marked as with traffic rules of high vehicle speed and large impact of congestion by accidents.

[0126] Optionally, the scenario classification method includes:

[0127] (1) Road feature classification based on GIS (Geographic Information System): The section attributes in the GIS data can be extracted and the scenario type (or road type) can be determined according to this attribute:

[0128]

[0129] (2) Based on historical vehicle speed distribution: Calculate the historical average speed of different roads :

[0130]

[0131] where T represents the historical duration participating in the calculation. If <30km / h, the road is more likely to belong to an urban road; if , the road is more likely to belong to a suburban road; if , then the road is more likely to belong to a highway.

[0132] After preprocessing operations such as data cleaning and noise reduction, and scene classification, the final multi-modal dataset is obtained :

[0133]

[0134] In the above formula are all datasets after preprocessing such as outlier handling, missing value filling, and data noise reduction. In subsequent operations, the data in also refers to the preprocessed data.

[0135] In summary, in this embodiment, multi-modal information such as GPS data of operating vehicles, mobile phone signaling data, traffic flow data of fixed monitoring points, meteorological data, and traffic event data is integrated in data collection, and traffic information from different sources is synthesized. Optimization is carried out in terms of synchronization, accuracy, coverage, etc. of data collection, so as to improve the accuracy and robustness of the subsequent model.

[0136] S120. Obtain the initial traffic network graph of the area to be predicted, where the traffic network graph takes roads as nodes and the connection relationships between roads as edges.

[0137] This step constructs the road network topology structure of the area to be predicted and abstracts the traffic network of this area into a graph structure. Specifically, the traffic state not only depends on time characteristics but is also affected by the spatial topology structure. For example, the congestion of road A may affect the traffic capacity of its neighboring road B, and this influence is restricted by factors such as road type and distance. Traditional time series prediction methods are difficult to model the spatial relationship between roads, while this embodiment uses a graph structure to accurately express the road connection relationship to enhance the understanding of local and global traffic states by the traffic prediction model in subsequent operations.

[0138] In a specific implementation manner, the traffic network of the area to be predicted can be abstracted into an undirected weighted graph :

[0139]

[0140] Among them, represents a node, and each road (or road section) is used as a node ; represents an edge. If there is a traffic connection between road and , then an edge is established; represents an adjacency matrix, which is used to describe the connection relationship and weight between nodes.

[0141] Optionally, each section of the area to be predicted is pre-divided. Based on the existing urban road electronic map database, each existing road segment (for example, a section of road between two intersections) can be regarded as a node to obtain road nodes that follow the real road network.

[0142] Optionally, the connection relationships of each road include the following types:

[0143] Direct connection: If two roads and are physically connected, an edge is established. For example, a long road is divided into multiple consecutive sections, and a direct connection relationship can be established between adjacent sections.

[0144] Intersection connection: If roads and are connected through an intersection (but there is no direct connection relationship), an edge is also established. For example, in an intersection, if the upper and lower (north-south) sections belong to the same road, they are directly connected; if the upper and lower (north-south) sections do not belong to the same road, it is an intersection connection; the upper and lower (north-south) and left and right (east-west) sections are intersection connections.

[0145] Highway ramp connection: An edge is established between the highway and its upper and lower ramps to reflect the traffic flow impact of the entrance / exit.

[0146] Optionally, the element in the th row and the th column of the adjacency matrix is the spatial correlation degree between road and road , which is used to characterize the degree of mutual influence of traffic between road and road and . The spatial correlation degree between adjacent roads

[0147]

[0148] can adopt an exponential decay form: where represents the geographical distance between road and road , and can calculate the straight-line Euclidean distance between the center points of the two sections as the geographical distance between the two sections according to the longitude and latitude of the starting point and the ending point of the section;

[0149] S130. Dynamically determine the adjacency matrix between roads according to the types of each road and the historical traffic flow of each road in the most recent period of time.

[0150] The traffic state change patterns of different types of roads are different. Therefore, in this embodiment, according to the road categories and historical traffic flow, the adjacency relationship between roads is dynamically adjusted, the influence weights between roads are optimized, and a traffic network topology structure suitable for different urban areas and traffic scenarios is formed, so that the subsequent traffic state prediction model can adaptively adjust the traffic state modeling strategy.

[0151] In a specific embodiment, the process of dynamically determining the adjacency matrix between roads may include the following steps:

[0152] Step 1. Estimate the spatial decay coefficient for maintaining the same traffic state between each two adjacent roads according to the historical traffic flow of each two adjacent roads in the most recent period of time. Optionally, estimate the distance decay coefficient in the adjacency matrix based on the least squares method to improve the self - adaptability of spatial modeling. For adjacent road pairs with the same or different road types, the following optimization objective functions are respectively used for parameter learning:

[0153]

[0154]

[0155] Among them, and respectively represent the historical traffic flows of road i and road j in the most recent period of time, represents the road and the road the geographical distance between them, represents the exponential function with the natural constant e as the base, represents the value of the distance decay coefficient, represents the value that can make the value in the parentheses the minimum; represents the distance decay coefficient when road i and road j are of the same type, represents the distance decay coefficient when road i and road j are of different types.

[0156] By introducing the exponential decay weight in the above formula, the closer road pairs have higher weights in the optimization, so as to strengthen the modeling of spatial local consistency; for road pairs with longer distances, even if there are traffic state differences, they will not significantly affect the loss function, thus effectively avoiding the interference of non - continuity caused by physical structure limitations to the model. Finally, by minimizing the sum of weighted error terms, the model can adaptively learn the most reasonable spatial decay scale , to capture the actual traffic impact intensity between roads and construct a more traffic - realistic dynamic adjacency relationship.

[0157] Step 2. According to the distance between every two adjacent roads and the distance decay coefficient, dynamically determine the mutual influence degree of traffic between every two adjacent roads as the elements of the adjacency matrix of the roads. Optionally, according to the following formula, dynamically determine the mutual influence degree of traffic between every two roads :

[0158]

[0159] where, represents the learnable cross - road - type influence factor (also called cross - scenario adjustment coefficient) for adjusting the spatial influence intensity between different types of roads. In the construction of the adjacency matrix: when the road types are the same, the influence degree is mainly determined by the distance, and the closer the distance, the greater the spatial influence; when the road types are different, in addition to considering the distance decay effect, is introduced to reflect the difference in traffic state propagation between different types of roads. The initial value of this parameter can be set to 0.5 and, as a learnable parameter during model training, is optimized through backpropagation to automatically learn the cross - scenario connection strength that best conforms to the actual traffic influence law.

[0160] In addition, since and are based on real - time or periodically updated historical traffic flow data, the distance decay coefficient re - estimated in each time window is also a dynamic variable. Furthermore, the spatial weight of each item in the adjacency matrix will also change dynamically with time, reflecting the adaptive modeling ability of this method for changes in the traffic network structure.

[0161] S140. In the initial traffic network graph, assign weights to each edge according to the adjacency matrix, and construct the current feature representation of each node according to the current multi - modal traffic data of each road to obtain a dynamic traffic network graph.

[0162] Based on the basic structure of the initial traffic network graph, this embodiment uses as the weight of edge , adaptively adjusts the adjacency relationship and spatial influence weights; at the same time, fuses traffic flow, vehicle speed, road geometric characteristics, weather, and traffic events to optimize the input feature representation of the nodes and obtain a complete traffic network graph.

[0163] In a specific implementation, in order to enable the subsequent GNN (Graph Neural Network) to fully learn the traffic state, for each road node Construct the input feature vector as follows:

[0164]

[0165] Wherein, represents the road at the moment input feature vector respectively represent the traffic volume and average speed of the road at the moment from the data of fixed monitoring points; represents the vehicle density per unit time passing through the road based on the GPS data of operating vehicles, used to make up for the monitoring blind area; represents the origin-destination intensity (OD intensity) of the area where the road is located, inferred from mobile phone signaling data, used to reflect the commuting tidal pressure of the area; represents the grade of the road such as arterial road, secondary arterial road, highway, etc.; represents the physical length of the road ; represents the meteorological data (such as rainfall, visibility, etc.) at the moment ; represents whether there is a sudden traffic event (such as an accident, road closure, etc.) on the road at the moment . By introducing the derived features constructed from the GPS data of operating vehicles and mobile phone signaling data, this embodiment can fully supplement the dynamic traffic perception information in the case of insufficient coverage of fixed monitoring points or no sensors installed on some roads, improve the modeling ability of the model for the actual urban traffic flow state, and enhance the overall prediction robustness.

[0166] Furthermore, in order to ensure that the input features have a similar numerical range, the maximum-minimum normalization method can be used to normalize each type of feature data so that all data are distributed in the interval [0,1].

[0167] S150. Input the dynamic traffic network graph into the GCN model and the GAT model respectively; the GCN model extracts the global spatial features according to the weights of each edge and the current feature representation of each node; the GAT model strengthens the role of key nodes to obtain the local attention spatial features.

[0168] This step is based on the adaptive traffic network topology graph constructed in S140, uses the graph neural network to model the spatial relationship between different roads, and extracts the spatial feature representation of each node at the current moment as the input basis for the subsequent time series modeling module.

[0169] Optionally, the combination of GCN (Graph Convolutional Network) and GAT (Graph Attention Network) can be adopted to enable the entire GNN model to capture both global topological information and enhance the modeling ability for key nodes (such as intersections and congested bottleneck sections). Among them, the GCN model can model the global topology and make the propagation of spatial information more stable; the GAT model can enhance the influence of key sections (such as intersections and highway ramps) and make the extraction of spatial features more targeted.

[0170] In a specific embodiment, the GCN model aggregates information through the adjacency matrix so that the features of each road node can absorb the information of neighboring roads:

[0171]

[0172] Among them, and respectively represent the current feature representations of each node in the th layer and the th layer of the GCN model (i.e., the feature representation at the current moment); represents the adjacency matrix with self-loops added, , represents the identity matrix of the same order; represents 's degree matrix; is a learnable parameter matrix; represents the non-linear activation function, such as ReLU (Rectified Linear Unit).

[0173] The role of the GCN model is to smooth the traffic state information and prevent the prediction results from fluctuating violently between adjacent roads; at the same time, it strengthens the global topological information and improves the model's understanding of the network structure.

[0174] In the GAT model, the adjacency matrix is used to define the neighbor set of nodes (only calculate the attention weights for connected node pairs) to prevent unconnected nodes from propagating information. Since different roads have different influences on the target road, an attention mechanism is introduced in the GAT model to adaptively assign different importance to each neighbor:

[0175]

[0176]

[0177] Among them, and respectively represent road i in the The current feature representation of layer and layer represents the set of neighbor nodes of road i, represents the layer of the GAT model, and the attention weight of node relative to node i in the layer; is the learnable feature mapping matrix of the layer of the GAT model, is the transpose of ; represents the concatenation operation of vectors or matrices (i.e., concatenating two vectors or matrices in a specific dimension into a new vector or matrix), and are both road indices in the set ; and represent roads and road respectively, and their current feature representations in the

[0178] By introducing a dynamic learnable attention weight mechanism, the GAT model can assign different influences to different neighbor nodes, rather than simply averaging neighbor information. If a neighbor (such as an intersection or a main road) has more significant or important feature changes, it will adaptively assign it a greater weight, thereby enhancing the influence of key sections (such as intersections and main roads) on the overall prediction and improving the modeling ability for complex traffic flow patterns (such as ramp merges and signal-controlled intersections).

[0179] Furthermore, to comprehensively utilize the global feature extraction ability of the GCN model and the local attention mechanism of the GAT model, this embodiment adopts a weighted fusion strategy to obtain the final spatial feature:

[0180]

[0181] where represents the road's final spatial feature at time ; represents the road's global spatial feature at time ; represents the road's local attention spatial feature at time and are both trainable weighted parameters.

[0182] In summary, S140 models the spatial correlation between roads based on GNN, depicts the diffusion patterns of vehicle flow and congestion in the road network, and combines the characteristics of GCN and GAT, taking into account both the global topological structure and the information of key road segments.

[0183] S160: For the time series of the multi-modal traffic data of each node before the to-be-predicted moment, perform short-term time series modeling and long-term time series modeling respectively to obtain short-term features and long-term features.

[0184] This step models the time dynamic changes of the traffic state to fully capture short-term fluctuations and long-term trends. Optionally, the short-term dynamic prediction can be performed through an LSTM (Long Short Term Memory) model to model short-term traffic flow fluctuations, such as congestion situations affected by traffic lights and accidents; the long-term trend learning can be performed through a Transformer model to extract periodic traffic patterns, such as morning and evening rush hours and weekend holiday travel trends.

[0185] In a specific embodiment, the time series modeling process may include the following steps:

[0186] Step 1: Construct the spatial feature time series to be processed. Optionally, the road at time time series is composed of the spatial feature representations at previous moments, and the formula is as follows:

[0187]

[0188] where represents the th spatial feature vector of the road obtained by fusing the GCN and GAT models. According to the needs of the modeling task, for each road at each moment, two spatial feature time series with different lengths can be constructed, which are used for short-term prediction and long-term modeling respectively.

[0189] Step 2: Input the shorter time series into the LSTM model for short-term time series modeling. The LSTM model can capture the dependency relationships over a longer time span better than the traditional RNN (Recurrent Neural Network). Specifically, for each road , the short-term time series features extracted by the LSTM can be expressed as:

[0190]

[0191] Among them, represents the short-term time series feature of the road at time ; represents the operation in the LSTM model.

[0192] Meanwhile, the longer time series is input into the Transformer model for long-term time series modeling... The Transformer needs to observe data over a longer time span to learn the periodic travel pattern rules, and the self-attention mechanism can efficiently model the global dependencies between different time steps. Specifically, for each road , the long-term time series feature extracted by the Transformer can be expressed as:

[0193]

[0194] Among them, represents the long-term time series feature of the road at time ; represents the operations in the Transformer model, mainly including:

[0195] Multi-head self-attention calculation:

[0196]

[0197] Among them, are the query, key, and value matrices respectively, all obtained by linear transformation from the input time series ; represents the similarity calculation between the query and the key; is the dimension of the key vector (for scaling); represents the normalized exponential function, used to calculate the attention weights ; the final output of the model is the value matrix after weighted summation , used to represent the long-term time series feature at the current moment.

[0198] S170. Predict the traffic state at the to-be-predicted moment according to the global spatial feature, local attention spatial feature, short-term feature, and long-term feature.

[0199] In this step, spatio-temporal feature fusion is first performed, fusing the spatial feature extracted by the GNN, the short-term time series feature extracted by the LSTM, and the long-term time series feature extracted by the Transformer to form a complete spatio-temporal feature representation, improving the prediction ability for emergencies and periodic patterns.

[0200] Optionally, the fused features can be expressed as:

[0201]

[0202] where represents the spatio-temporal fused features of road at time , , and are all learnable fusion weights, and their values are continuously adjusted during training to minimize the prediction error. The model will automatically learn the relative importance of spatial features, short-term features, and long-term features in different scenarios.

[0203] After feature fusion, traffic state prediction and intelligent decision support can be performed based on the spatio-temporal fused features. Optionally, end-to-end learning is performed through a multi-layer perceptron to output the traffic flow, vehicle speed, and congestion index at future times, and this process can be expressed as:

[0204]

[0205] where represents the traffic state of road at time , including traffic flow, vehicle speed, and congestion index, represents the operation in the multi-layer perceptron.

[0206] Furthermore, the above dynamic traffic network diagram, GCN model, GAT model, LSTM model, and Transformer model together constitute a complete traffic state prediction model, and the parameters of each part in the model can be jointly determined through joint training. Optionally, a supervision loss (such as mean square error, cross entropy, etc.) can be constructed between the model prediction result of the traffic state and the real traffic state, and the complete loss function is jointly constituted by this supervision loss and the polynomial to be minimized in the above least squares method, so as to complete the model training; alternatively, the above least squares method can be separated from the model training, the optimal distance decay coefficient can be determined in advance through the least squares method, and then the model is trained based on this coefficient. At this time, the loss function only includes the above supervision loss, and the distance decay coefficient is updated according to the latest flow every once in a while, and the model is fine-tuned once. The adjustable parameters in the entire prediction model are jointly optimized through gradient backpropagation, and their parameter updates follow the following formula:

[0207]

[0208] where represents all the learnable parameters in the entire prediction model, represents the updated parameters, denotes the learning rate, is the loss function value.

[0209] Based on the trained model, the prediction results of future traffic states can be obtained, providing data support for signal light optimization, route guidance, public transportation resource allocation, etc., and improving the intelligent level of traffic management. Specifically, in terms of signal light optimization, the signal light cycle and phase time can be adjusted based on future traffic flow prediction to improve road passing capacity; in terms of route guidance, the predicted congestion index can be used to provide optimized route recommendations for the navigation system to reduce the risk of traffic congestion; in terms of public transportation resource allocation, the bus, taxi, and online car-hailing dispatching can be optimized by combining the predicted passenger flow tidal effect to improve transportation efficiency.

[0210] In summary, this embodiment provides a traffic state prediction method based on multi-modal data and adaptive topology modeling, which integrates operating vehicle GPS data, mobile phone signaling data, fixed monitoring point data, meteorological data, and traffic event data, and optimizes the data quality through anomaly detection, data denoising, missing value filling, etc., providing a good data foundation for traffic state prediction. In traffic state prediction, this embodiment constructs an adaptively changing adjacency matrix based on road categories and historical traffic flows, dynamically adjusts the traffic network topology, enabling it to adjust the influence weights between roads over time; subsequently, it is applied to the spatial feature extraction model, and the global topology information of the road network is dynamically extracted through the GCN model, and the dependence relationship between key roads is strengthened by using the multi-layer attention mechanism of the GAT model; at the same time, the optimized multi-modal data time series are respectively input into the LSTM and Transformer models to model short-term emergencies and long-term travel patterns respectively, further improving the prediction accuracy; finally, the global spatial features, local attention spatial features, short-term time series features, and long-term time series features are fused and input into a multi-layer perceptron for regression calculation to output future traffic flow, vehicle speed, and congestion index. This embodiment effectively improves the accuracy and robustness of traffic state prediction through adaptive dynamic topology modeling, graph neural network optimization, and time series feature fusion, and can be applied to intelligent traffic management, signal optimization, road planning, and congestion control, providing more efficient data support.

[0211] It should be noted that the user data involved in this application (including operating vehicle GPS data, mobile phone signaling data, etc.) are all information and data authorized by users or fully authorized by all parties, and the collection, use, and processing of relevant data need to comply with the relevant laws, regulations, and standards of relevant countries and regions, and corresponding operation entrances are provided for users to choose to authorize or refuse.

[0212] Figure 3 is a schematic structural diagram of an electronic device provided by an embodiment of the present invention, as Figure 3As shown, the device includes a processor 60, a memory 61, an input device 62, and an output device 63; the number of processors 60 in the device can be one or more, Figure 3 and one processor 60 is taken as an example herein; the processor 60, the memory 61, the input device 62, and the output device 63 in the device can be connected through a bus or other means, Figure 3 and taking connection through a bus as an example herein.

[0213] The memory 61, as a computer-readable storage medium, can be used to store software programs, computer-executable programs, and modules, such as program instructions / modules corresponding to the traffic state prediction method based on multimodal data and adaptive topology modeling in the embodiments of the present invention. The processor 60 executes various functional applications and data processing of the device by running the software programs, instructions, and modules stored in the memory 61, that is, implements the above-mentioned traffic state prediction method based on multimodal data and adaptive topology modeling.

[0214] The memory 61 mainly includes a program storage area and a data storage area. Among them, the program storage area can store an operating system and application programs required for at least one function; the data storage area can store data created according to the use of the terminal, etc. In addition, the memory 61 can include high-speed random access memory, and can also include non-volatile memory, such as at least one magnetic disk storage device, a flash memory device, or other non-volatile solid-state storage devices. In some instances, the memory 61 can further include a memory remotely set relative to the processor 60, and these remote memories can be connected to the device through a network. Examples of the above network include but are not limited to the Internet, an enterprise intranet, a local area network, a mobile communication network, and combinations thereof.

[0215] The input device 62 can be used to receive input digital or character information, and generate key signal inputs related to the user settings and function control of the device. The output device 63 can include a display device such as a display screen.

[0216] The embodiments of the present invention also provide a computer-readable storage medium, on which a computer program is stored, and when the program is executed by a processor, it implements the traffic state prediction method based on multimodal data and adaptive topology modeling in any embodiment.

[0217] The computer storage medium of the embodiments of the present invention may adopt any combination of one or more computer-readable media. The computer-readable media may be computer-readable signal media or computer-readable storage media. The computer-readable storage media may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the above. More specific examples (a non-exhaustive list) of the computer-readable storage media include: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this document, the computer-readable storage media may be any tangible medium that contains or stores a program, and the program may be used by or in combination with an instruction execution system, apparatus, or device.

[0218] The computer-readable signal media may include data signals propagated in a baseband or as part of a carrier wave, which carry computer-readable program codes. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. The computer-readable signal media may also be any computer-readable media other than the computer-readable storage media, and the computer-readable media may send, propagate, or transmit a program for use by or in combination with an instruction execution system, apparatus, or device.

[0219] The program codes contained on the computer-readable media may be transmitted by any appropriate media, including but not limited to wireless, wire, optical cable, RF, etc., or any suitable combination of the above.

[0220] The computer program codes for performing the operations of the present invention may be written in one or more programming languages or combinations thereof. The programming languages include object-oriented programming languages such as Java, Smalltalk, C++, and also include conventional procedural programming languages such as C language or similar programming languages. The program codes may be executed entirely on the user's computer, partially on the user's computer, executed as an independent software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (for example, by using an Internet service provider to connect through the Internet).

[0221] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions described in the foregoing embodiments, or perform equivalent replacements on some or all of the technical features; and these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the technical solutions of the embodiments of the present invention.

Claims

1. A traffic state prediction method based on multi-modal data and adaptive topology modeling, characterized in that Including: Obtain an initial traffic network graph of the area to be predicted, where the traffic network graph takes roads as nodes and the connection relationships between roads as edges; Dynamically determine the adjacency matrix between roads according to the types of each road and the historical traffic flow of each road in the most recent period of time, where each element in the adjacency matrix is used to represent the degree of mutual influence of traffic between two roads; In the initial traffic network graph, assign weights to each edge according to the adjacency matrix, and construct the current feature representation of each node according to the current multimodal traffic data of each road to obtain a dynamic traffic network graph; Input the dynamic traffic network graph into the GCN model and the GAT model respectively; the GCN model extracts global spatial features according to the weights of each edge and the current feature representation of each node; the GAT model strengthens the role of key nodes to obtain local attention spatial features; Perform short-term time series modeling and long-term time series modeling on the time series of each node's multimodal traffic data in the most recent period of time respectively to obtain short-term features and long-term features; Predict the future traffic state according to the global spatial features, local attention spatial features, short-term features and long-term features.

2. The method according to claim 1, wherein The dynamically determining the adjacency matrix between roads according to the types of each road and the historical traffic flow of each road in the most recent period of time includes: Estimate the distance decay coefficient for maintaining consistent traffic flow between each two adjacent roads according to the historical traffic flow of each two adjacent roads in the most recent period of time; Dynamically determine the degree of mutual influence of traffic between each two adjacent roads according to the distance between each two adjacent roads and the distance decay coefficient as the elements of the adjacency matrix between roads.

3. The method according to claim 2, wherein The estimating the distance decay coefficient for maintaining consistent traffic flow between each two adjacent roads according to the historical traffic flow of each two adjacent roads in the most recent period of time includes: Based on the least squares method, estimate the distance decay coefficient that keeps the traffic flow consistent between every two adjacent roads : ， Among them, indicates that road i and road j are two adjacent roads, and respectively represent the historical traffic flows of road i and road j in the most recent period of time, represents the geographical distance between road i and road j, represents the exponential function with the natural constant e as the base, means that takes the minimum value of value.

4. The method according to claim 2, wherein The dynamically determining the degree of mutual influence of traffic between each two adjacent roads according to the distance between each two adjacent roads and the distance decay coefficient includes: Dynamically determine the mutual influence degree of traffic between every two roads according to the following formula :[[]]END]] ， Among them, represents the exponential function with the natural constant e as the base, represents the geographical distance between road i and road j, and represent the types of road i and road j respectively, indicates that road i and road j are two adjacent roads, represents the distance attenuation coefficient between adjacent roads of the same type, represents the distance attenuation coefficient between adjacent roads of different types, represents a learnable cross-road type factor.

5. The method according to claim 1, characterized in that The GCN model extracting global spatial features according to the weights of each edge and the current feature representation of each node includes: The GCN model extracts global spatial features according to the following formula: ， Among them, and respectively represent the current feature representations of each road layer and layer in the GCN model; represents the adjacency matrix with self-loops added, , represents the adjacency matrix, represents the identity matrix of the same order; represents 's degree matrix; represents the learnable parameter matrix; represents the non-linear activation function.

6. The method according to claim 1, characterized in that The GAT model strengthening the role of key nodes to obtain local attention spatial features includes: Determine the neighbor nodes of each node according to the weights of each edge; The GAT model strengthens the role of key nodes according to the following formula to obtain local attention spatial features: ， ， Among them, and respectively represent the current feature representations of road i at the -th layer and the -th layer of the GAT model. represents the non-linear activation function. represents the set of neighbor nodes of road i. represents the attention weight of road j relative to road i in the -th layer of the GAT model. is the learnable feature mapping matrix of the -th layer of the GAT model. is the learnable vector of the attention coefficient. is transposed. represents the concatenation operation of vectors or matrices. represents the leaky rectified linear unit function. represents the exponential function with the natural constant e as the base. Both i and j represent the road indices in the set . and respectively represent the current feature representations of road k and road j at the -th layer of the GAT model.

7. The method according to claim 1, wherein The obtaining the initial traffic network graph of the area to be predicted includes: Determine the connection relationship between two roads with the same direction and physical connection as a direct connection; Determine the connection relationship between two intersection roads with different directions as an intersection connection; Determine the connection relationship between a highway and its on- and off-ramps as a highway ramp connection.

8. The method according to claim 1, characterized in that, The constructing the current feature representation of each node according to the current multimodal traffic data of each road includes: Obtain the current operating vehicle GPS data, mobile phone signaling data, fixed monitoring point traffic flow data, meteorological data and traffic event data of the area to be predicted; Based on the acquired data, determine the current traffic flow, vehicle speed, road grade, weather characteristics, and traffic event characteristics of each road within the area; Based on the current data and road grade determined for each road, construct the current feature representation of each node.

9. An electronic device, characterized in that, Including: One or more processors; A memory for storing one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors implement the traffic state prediction method based on multi-modal data and adaptive topology modeling according to any one of claims 1-8.

10. A computer-readable storage medium, characterized in that, Stored thereon is a computer program which, when executed by a processor, implements the traffic state prediction method based on multi-modal data and adaptive topology modeling according to any one of claims 1-8.

Images