Traffic state prediction method based on multimodal data and adaptive topology modeling

Abstract

An embodiment of the present invention discloses a traffic state prediction method based on multimodal data and adaptive topology modeling. The method includes: dynamically determining the adjacency matrix between roads based on the type and historical traffic flow of each road, and inputting the adjacency matrix into a GCN model and a GAT model respectively based on the dynamic traffic network diagram; the GCN model extracts global spatial features based on the weight of each edge and the current feature representation of each node; the GAT model strengthens the role of key nodes to obtain local attention spatial features; short-term time series modeling and long-term time series modeling are performed on the time series of multimodal traffic data of each node in the most recent period, respectively, to obtain short-term features and long-term features; and predicting future traffic states based on the global spatial features, local attention spatial features, short-term features, and long-term features. This embodiment improves the accuracy of traffic state prediction.

Description

Technical Field

[0001] The embodiments of the present invention relate to the fields of intelligent transportation and artificial intelligence technology, and in particular to a traffic state prediction method based on multimodal data and adaptive topology modeling. Background Art

[0002] With the development of intelligent transportation systems, urban traffic prediction technology has become one of the key technologies for improving road traffic efficiency and optimizing traffic management strategies. Currently, the main traffic prediction methods can be divided into the following categories:

[0003] 1. Statistical modeling-based methods: Time series analysis methods such as ARIMA (Auto Regressive Integrated Moving Average) and SARIMA (Seasonal Auto Regressive Integrated Moving Average) use historical traffic data for prediction. However, their linear assumptions limit their ability to model complex nonlinear traffic flows and make it difficult to capture the spatial dependencies between roads.

[0004] 2. Graph neural network based on fixed adjacency matrix: This approach uses graph neural network and other methods for spatial modeling. However, the use of a static adjacency matrix cannot dynamically adapt to different road types and real-time traffic flow changes, resulting in limited prediction accuracy.

[0005] 3. Traffic prediction based on time series deep learning: This method only models traffic changes over time and fails to effectively consider the spatial interactions between roads, making it difficult to accurately predict traffic anomalies.

[0006] Patent application CN119274345A provides a traffic flow prediction method based on spatiotemporal correlation of deep learning, and patent application CN116071923A provides a traffic flow prediction method based on adaptive graph fusion convolutional network, but they are also unable to solve the above problems well. Summary of the Invention

[0007] An embodiment of the present invention provides a traffic status prediction method based on multimodal data and adaptive topology modeling to solve at least one of the above technical problems.

[0008] In a first aspect, an embodiment of the present invention provides a traffic state prediction method based on multimodal data and adaptive topology modeling, comprising:

[0009] Obtain an initial traffic network diagram of the area to be predicted, wherein the traffic network diagram uses roads as nodes and the connections between roads as edges;

[0010] Based on the type of each road and the historical traffic flow of each road in the recent period, the adjacency matrix between roads is dynamically determined, where each element in the adjacency matrix is ​​used to represent the degree of mutual influence between the traffic of two roads;

[0011] In the initial traffic network graph, weights are assigned to each edge according to the adjacency matrix, and a current feature representation of each node is constructed according to the current multimodal traffic data of each road to obtain a dynamic traffic network graph;

[0012] The dynamic traffic network graph is input into the GCN model and the GAT model respectively; the GCN model extracts global spatial features based on the weight of each edge and the current feature representation of each node; the GAT model strengthens the role of key nodes to obtain local attention spatial features;

[0013] For the time series of multimodal traffic data of each node in the most recent period, short-term time series modeling and long-term time series modeling are performed respectively to obtain short-term features and long-term features;

[0014] The future traffic status is predicted based on the global spatial features, local attention spatial features, short-term features and long-term features.

[0015] In a second aspect, an embodiment of the present invention provides an electronic device, comprising:

[0016] one or more processors;

[0017] a memory for storing one or more programs,

[0018] When the one or more programs are executed by the one or more processors, the one or more processors implement the traffic status prediction method based on multimodal data and adaptive topology modeling described in any embodiment.

[0019] In a third aspect, an embodiment of the present invention further provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the traffic status prediction method based on multimodal data and adaptive topology modeling described in any embodiment.

[0020] In summary, the present invention provides a traffic state prediction method based on multimodal data and adaptive topology modeling. This method integrates multimodal traffic data and optimizes data quality, providing a sound data foundation for traffic state prediction. In traffic state prediction, this method constructs an adaptively changing adjacency matrix based on road categories and historical traffic flows, dynamically adjusting the traffic network topology so that the influence weights between roads can be adjusted over time. This matrix is ​​then applied to a spatial feature extraction model, dynamically extracting global topological information from the road network using the GCN model and leveraging the multi-layer attention mechanism of the GAT model to strengthen the dependencies between key roads. Simultaneously, the optimized multimodal data time series are fed into LSTM and Transformer models, respectively, to model short-term emergencies and long-term travel patterns, further improving prediction accuracy. Finally, the global spatial features, local attention spatial features, short-term temporal features, and long-term temporal features are fused and fed into a multi-layer perceptron for regression calculation, outputting future traffic flow, vehicle speed, and congestion index. This embodiment effectively improves the accuracy and robustness of traffic status prediction through adaptive dynamic topology modeling, graph neural network optimization, and time series feature fusion. It can be applied to intelligent traffic management, signal optimization, road planning, and congestion control, providing more efficient data support. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] In order to more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the specific embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0022] Figure 1 This is a flow chart of a traffic state prediction method based on multimodal data and adaptive topology modeling provided by an embodiment of the present invention;

[0023] Figure 2 is a flowchart of another traffic state prediction method based on multimodal data and adaptive topology modeling provided by an embodiment of the present invention;

[0024] Figure 3 A schematic structural diagram of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0025] To make the objectives, technical solutions, and advantages of the present invention more clear, the technical solutions of the present invention are described clearly and completely below. Obviously, the embodiments described are only some of the embodiments of the present invention, not all of them. All other embodiments derived by persons of ordinary skill in the art based on the embodiments of the present invention without inventive effort are also within the scope of protection of the present invention.

[0026] In the description of the present invention, it should be noted that the terms "center," "upper," "lower," "left," "right," "vertical," "horizontal," "inner," and "outer," etc., indicating orientations or positional relationships, are based on the orientations or positional relationships shown in the accompanying drawings and are intended solely to facilitate and simplify the description of the present invention. They are not intended to indicate or imply that the devices or components referred to must have, be constructed, or operate in a specific orientation, and therefore should not be construed as limitations on the present invention. Furthermore, the terms "first," "second," and "third" are used for descriptive purposes only and should not be construed as indicating or implying relative importance.

[0027] In the description of the present invention, it should also be noted that, unless otherwise expressly specified or limited, the terms "mounted," "connected," and "connected" should be understood broadly. For example, they may refer to fixed, detachable, or integral connections; mechanical or electrical connections; direct or indirect connections through an intermediate medium; and internal communication between two components. Those skilled in the art will understand the specific meanings of the above terms in the present invention based on the specific circumstances.

[0028] Figure 1 This is a flow chart of a traffic state prediction method based on multimodal data and adaptive topology modeling provided by an embodiment of the present invention. This method achieves accurate urban traffic state prediction through multimodal data fusion and adaptive traffic network topology, and can be widely used in application scenarios such as urban traffic flow prediction, signal light optimization, road planning and congestion management. The method is executed by an electronic device, such as Figure 1 As shown, specifically including:

[0029] S110 . For the area to be predicted, collect GPS data, mobile phone signaling data, fixed monitoring point data, meteorological data, and traffic event data of operating vehicles in the area.

[0030] This embodiment first collects and processes multimodal data related to the traffic status of the urban road network, aiming to provide a high-quality data foundation for the subsequent traffic prediction model.

[0031] Figure 2This is a flow chart of another traffic state prediction method based on multimodal data and adaptive topology modeling provided by an embodiment of the present invention, which shows the key links of the method of this embodiment from another perspective. Figure 1 and Figure 2 In this embodiment, for a future moment to be predicted, multimodal data is first collected and processed for a period of time ending at the moment. The data sources may include:

[0032] (1) GPS (Global Positioning System) data for operating vehicles

[0033] Data source: Mainly from long-distance passenger transport, taxis, buses, online ride-hailing services, etc. equipped with GPS devices.

[0034] Sampling frequency: usually 1 second to 30 seconds (adjustable).

[0035] Main fields:

[0036]

[0037] in:

[0038] Indicates vehicle At the moment GPS data;

[0039] and Represents vehicles At the moment longitude and latitude;

[0040] Indicates vehicle At the moment Speed ​​(unit: km / h, kilometers per hour);

[0041] Indicates vehicle At the moment The heading angle;

[0042] Indicates vehicle At the moment The vehicle status (such as loaded / empty, running / stopped).

[0043] GPS data from commercial vehicles is high-frequency and can reflect microscopic changes in road traffic flow, such as queues and traffic light activation. This data can be used to infer trajectories, calculate local road traffic flow, and assist in accident detection and signal optimization.

[0044] (2) Mobile phone signaling data

[0045] Data source: Anonymized mobile phone base station location information provided by mobile operators.

[0046] Update frequency: Sampling is performed every 5-10 minutes (depending on the user's mobility status and base station switching frequency).

[0047] Main fields:

[0048]

[0049] in:

[0050] Mobile phone users At the moment Mobile phone signaling data;

[0051] Mobile phone users At the moment The ID of the connected base station;

[0052] Record signaling timestamp.

[0053] Mobile phone signaling data has extensive coverage and can supplement GPS data with information about private vehicles and pedestrians. It can be used for origin-destination (OD) analysis, calculating origin-destination distribution and identifying crowd tidal effects.

[0054] (3) Traffic flow data at fixed monitoring points

[0055] Data sources: fixed monitoring equipment such as ground sensing coils, microwave radars, and high-definition cameras on highways and urban main roads.

[0056] Sampling frequency: Updated every 1-5 minutes (device configurable).

[0057] Main fields:

[0058] :Indicates road At the moment Traffic volume (unit: vehicles / hour);

[0059] :Indicates road At the moment The average vehicle speed (unit: km / h).

[0060] Traffic flow data from fixed monitoring points provides long-term, stable data that can be combined with mobile data for calibration. Fixed monitoring point traffic flow data can be used to supplement potential sampling biases in individual behavior data (such as GPS data from operating vehicles and mobile phone signaling data).

[0061] (4) Meteorological data

[0062] Data sources: National Meteorological Administration API (Application Programming Interface), commercial meteorological data services.

[0063] Sampling frequency: updated every hour.

[0064] Main fields:

[0065]

[0066] in:

[0067] Indicates time Weather data

[0068] Indicates time rainfall (mm);

[0069] Indicates time Temperature (℃);

[0070] Indicates time Visibility (m).

[0071] Traffic is significantly affected by weather. Rainy and snowy days can reduce vehicle speeds and increase accident rates. Combining meteorological data with other data can help analyze the impact of weather on road networks and improve forecast accuracy.

[0072] (5) Traffic incident data

[0073] Data sources: Traffic management department accident reporting system, real-time event reports on social media.

[0074] Sampling frequency: Real-time update.

[0075] Main fields:

[0076]

[0077] in:

[0078] Indicates time Traffic incident data;

[0079] Indicates time Type of traffic event (accident, construction, road closure, etc.);

[0080] Indicates time Location of the traffic incident;

[0081] Indicates time Traffic severity (0-10).

[0082] Traffic event data can predict abnormal congestion, such as decreased traffic flow in construction zones and accident-induced queues. Traffic event data can also be used for causal reasoning and assessing the impact of events on traffic.

[0083] As mentioned above, this embodiment takes into account high-precision trajectory (GPS), macro travel mode (signaling) and long-term monitoring (fixed monitoring points), providing comprehensive congestion inducements (meteorological + events), laying the data foundation for subsequent modeling. After data collection, through time alignment, deduplication, and fusion, the original multimodal data set is formed. :

[0084]

[0085] in, and Respectively 、 、 、 、 、 、 and The sets obtained by taking the union of all vehicles, mobile phone users, roads and / or time respectively.

[0086] Then, the original multimodal dataset Cleaning, noise reduction, and scene classification are performed to improve data quality and ensure the accuracy and robustness of subsequent modeling. In a specific embodiment, this process may include the following steps:

[0087] Step 1: Detect and process outliers: Eliminate sensor errors and extreme outliers from the data. Optional sources of data anomalies include: device errors, such as GPS sensors that may produce erroneous values ​​due to malfunctions, such as unreasonable speeds (>300 km / h); environmental interference, such as mobile phone signaling data that may be affected by base station switching, causing position drift; and human factors, such as incorrect data input or loss. Optional outlier detection methods include:

[0088] (1) Z-score method

[0089] This method is applicable to numerical data (such as traffic volume, vehicle speed) and calculates the standardized score of the data. :

[0090]

[0091] in, Represents the data value of the current feature, represents the mean of the feature, Indicates the standard deviation of the feature. If |Z| > 3, it is considered an outlier (a point outside of 3 times the standard deviation).

[0092] (2) Boxplot method

[0093] This method is applicable to data such as flow, velocity, and precipitation, and calculates the interquartile range. :

[0094]

[0095] in, represents the 25th percentile, Indicates the 75th percentile. If the data value ,or , then it is believed that It is an outlier.

[0096] (3) Speed ​​jump detection

[0097] This method is applicable to GPS data and calculates the speed change rate of two consecutive GPS points of the same vehicle. :

[0098]

[0099] in, and Represent the speed of the same vehicle at two adjacent moments extracted from GPS data. If a threshold is set (such as 50km / h / s), it will be considered as abnormal data.

[0100] (4) Signaling data anomaly detection

[0101] If the user travels more than 100 kilometers within 5 minutes, the signaling point may be affected by base station switching and needs to be eliminated.

[0102] Step 2: Fill missing values: Ensure data integrity and prevent the model from misjudging due to missing data. Missing data may lead to misjudgment of the prediction model. This embodiment uses different methods to fill different types of missing data, as shown in Table 1:

[0103] Table 1

[0104] Data Type Filling method Applicable Scenarios Traffic flow#timg# Linear interpolation Short-term traffic loss Speed#timg# Linear interpolation Speed ​​fluctuations between measurement points GPS tracks Bezier curve interpolation Track breakpoint filling Mobile phone signaling Nearest neighbor interpolation 5-10 minutes gap filling Weather data Historical mean replacement Short-term weather data loss

[0105] Specifically, (1) Linear interpolation: Applicable to short-term missing time series:

[0106]

[0107] in, represents the data value at the missing moment, and Represents the data values ​​of the two moments before and after the missing moment respectively.

[0108] (2) Bezier curve interpolation: Applicable to GPS trajectory:

[0109]

[0110] in, represents missing data points in the trajectory, 、 and They represent the reference point before the missing point in the trajectory, the estimated control point at the missing point, and the reference point after the missing point. Represents the normalized time progress parameter (usually obtained by linear mapping of time, in the range [0,1]).

[0111] (3) Nearest neighbor interpolation: Applicable to mobile phone signaling:

[0112]

[0113] in, represents the data value at the missing moment, Indicates the data value at the moment before the missing moment.

[0114] Step 3: Perform data denoising: Reduce measurement errors and random jitter to extract true traffic trends. Optional data denoising methods include:

[0115] (1) Sliding average method: applicable to traffic flow and speed data.

[0116]

[0117] in, Represents the original data, represents the smoothed data, Indicates time The length of the sliding time window centered at . When the price is higher than the target price, it can smooth out short-term fluctuations.

[0118] (2) Kalman filter: applicable to GPS trajectories.

[0119]

[0120] in, and Respectively represent the data values ​​of two adjacent moments after filtering, and They represent the state transfer matrix in Kalman filtering, represents Gaussian noise, Represents the control input, such as the acceleration or direction change of the vehicle.

[0121] It should be noted that among the above-mentioned data preprocessing methods, some different methods use the same variables. In this case, the meaning of each variable should be treated differently according to the specific description of the variable in the method.

[0122] Step 4: Scenario Classification: Rationally group the data according to road characteristics and traffic patterns to support the adaptive prediction model. Optionally, the scenario classification in this embodiment can be understood as road type classification. Different road types have different traffic patterns, so it can be divided into the following scenarios:

[0123] Urban roads: marked as ,The traffic rules are that there are many traffic lights and low speed;

[0124] Suburban roads: marked as ,The traffic pattern is that there are few traffic lights and the speed is relatively stable;

[0125] Highway: marked as ,The traffic rules are high speed, congestion and great impact of accidents.

[0126] Optionally, the scene classification method includes:

[0127] (1) Road feature classification based on GIS (Geographic Information System): road segment attributes can be extracted from GIS data , and determine the scene type based on this attribute (or road type):

[0128]

[0129] (2) Based on historical speed distribution: Calculate the historical average speed of different roads :

[0130]

[0131] Among them, T represents the historical length of time involved in the calculation. <30km / h, then the road More likely to be an urban road; if , then the road More likely to be a suburban road; if , then the road More likely to be a highway.

[0132] After data cleaning, noise reduction and other preprocessing operations and scene classification, the final multimodal dataset is obtained :

[0133]

[0134] In the above formula All of them are pre-processed data sets after outlier processing, missing value filling and data noise reduction. The data in the figure also refers to the preprocessed data.

[0135] In summary, this embodiment integrates multimodal information such as operating vehicle GPS data, mobile phone signaling data, traffic flow data from fixed monitoring points, meteorological data, and traffic event data in terms of data collection, integrates traffic information from different sources, and optimizes the synchronization, accuracy, coverage, etc. of data collection, thereby improving the accuracy and robustness of subsequent models.

[0136] S120: Obtain an initial traffic network diagram of the area to be predicted, wherein the traffic network diagram uses roads as nodes and connections between roads as edges.

[0137] This step constructs the road network topology for the area to be predicted, abstracting the area's traffic network into a graph structure. Specifically, traffic conditions depend not only on temporal characteristics but also on spatial topology. For example, congestion on Road A may affect the capacity of adjacent Road B, and this impact is constrained by factors such as road type and distance. Traditional time series prediction methods struggle to model the spatial relationships between roads. However, this embodiment uses a graph structure to accurately represent road connectivity, enhancing the traffic prediction model's understanding of local and global traffic conditions in subsequent operations.

[0138] In a specific embodiment, the traffic network of the area to be predicted can be abstracted as an undirected weighted graph :

[0139]

[0140] in, Represents a node, each road (or road segment) As a node ; Represents an edge, if the road and When there is a traffic connection between ; Represents the adjacency matrix, which is used to describe the connection relationship and weight between nodes.

[0141] Optionally, the road sections in the area to be predicted are pre-divided. Based on an existing urban road electronic map database, each existing road segment (such as a section of road between two intersections) can be regarded as a node to obtain road nodes that follow the actual road network.

[0142] Optionally, the connection relationships between roads include the following:

[0143] Directly connected: If two roads and If they are physically connected, an edge is established. For example, a long road is divided into multiple continuous sections, and direct connections can be established between adjacent sections.

[0144] Intersection connection: If the road and Edges are also created for intersections that connect (but not directly connect) with each other. For example, at an intersection, if the upper and lower (north-south) sections belong to the same road, they are directly connected. If the upper and lower (north-south) sections do not belong to the same road, they are connected by an intersection. The upper and lower (north-south) sections and the left and right (east-west) sections are connected by an intersection.

[0145] Highway ramp connection: Create edges between highways and their on- and off-ramps to reflect the impact of entrance / exit traffic.

[0146] Optionally, the adjacency matrix Rank Elements of a column For the road and roads The spatial correlation between To characterize roads and roads The degree of mutual influence between traffic. and The spatial correlation between can be expressed in an exponential decay form:

[0147]

[0148] in, Indicates road and The geographical distance between the two sections can be calculated by calculating the straight-line Euclidean distance between the center points of the two sections based on the longitude and latitude of the starting point and the end point of the section. Indicates the scene-adaptive distance attenuation coefficient (different values ​​are set for different road types).

[0149] S130 , dynamically determining an adjacency matrix between roads based on the type of each road and the historical traffic flow of each road in a recent period of time.

[0150] Different types of roads have different traffic status change patterns. Therefore, this embodiment dynamically adjusts the adjacency relationship between roads based on road categories and historical traffic flows, optimizes the influence weights between roads, and forms a traffic network topology that adapts to different urban areas and traffic scenarios. This allows subsequent traffic status prediction models to adaptively adjust traffic status modeling strategies.

[0151] In a specific embodiment, the process of dynamically determining the road adjacency matrix may include the following steps:

[0152] Step 1: Based on the historical traffic flow of each pair of adjacent roads over a recent period of time, estimate the spatial attenuation coefficient that maintains consistent traffic conditions between each pair of adjacent roads. Optionally, estimate the distance attenuation coefficient in the adjacency matrix based on the least squares method to improve the adaptability of spatial modeling. For adjacent road pairs of the same or different road types, the following optimization objective function is used for parameter learning:

[0153]

[0154]

[0155] in, and Represent the historical traffic flow of road i and road j in the recent period, Indicates road and roads The geographical distance between represents the exponential function with the natural constant e as the base, Indicates the value of the distance attenuation coefficient, Indicates that the minimum value in the brackets can be obtained value; represents the distance attenuation coefficient when roads i and j are of the same type, Indicates the distance attenuation coefficient when road i and road j are of different types.

[0156] The above formula introduces exponential decay weight , so that road pairs with closer distances have higher weights in the optimization, thereby strengthening the modeling of spatial local consistency; while road pairs with farther distances, even if there are differences in traffic conditions, will not significantly affect the loss function, thus effectively avoiding the interference of discontinuities caused by physical structure limitations on the model. Ultimately, by minimizing the sum of weighted error terms, the model can adaptively learn the most reasonable spatial attenuation scale. , in order to capture the actual traffic impact intensity between roads and construct a dynamic adjacency relationship that is more in line with traffic reality.

[0157] Step 2: Based on the distance between each two adjacent roads and the distance attenuation coefficient, dynamically determine the degree of mutual influence of traffic between each two adjacent roads, and use it as an element of the road adjacency matrix. Optionally, dynamically determine the degree of mutual influence of traffic between each two roads according to the following formula: :

[0158]

[0159] in, Represents a learnable cross-road type influence factor (also known as a cross-scenario adjustment coefficient), which is used to adjust the spatial influence intensity between different types of roads. In the construction of the adjacency matrix: when the road types are the same, the degree of influence is mainly determined by the distance. The closer the distance, the greater the spatial influence. When the road types are different, in addition to considering the attenuation effect of distance, we also introduce This parameter is initially set to 0.5 to reflect the differences in traffic state propagation between different types of roads. It is used as a learnable parameter during model training and optimized through backpropagation to automatically learn the cross-scenario connection strength that best reflects the actual traffic impact patterns.

[0160] In addition, due to and It is based on real-time or periodically updated historical traffic flow data, so the distance decay coefficient is re-estimated in each time window. is also a dynamic variable. Furthermore, the spatial weight of each item in the adjacency matrix It will also change dynamically over time, reflecting the adaptive modeling capability of this method for changes in the transportation network structure.

[0161] S140. In the initial traffic network graph, weights are assigned to each edge according to the adjacency matrix, and a current feature representation of each node is constructed according to the current multimodal traffic data of each road to obtain a dynamic traffic network graph.

[0162] This embodiment is based on the basic structure of the initial traffic network diagram. As an edge The weight of each node is used to adaptively adjust the adjacency relationship and spatial influence weight; at the same time, the traffic volume, speed, road geometry, weather and traffic events are integrated to optimize the input feature representation of the node and obtain a complete traffic network diagram.

[0163] In a specific embodiment, in order to allow the subsequent GNN (Graph Neural Network) to fully learn the traffic status, each road node can be Construct the following input feature vector:

[0164]

[0165] in, Indicates road At the moment The input feature vector Represents roads At the moment Traffic volume and average speed are derived from data at fixed monitoring points; Indicates the number of times a vehicle crosses a road per unit time based on GPS data statistics of an operating vehicle. Vehicle density is used to fill monitoring blind spots; Represents roads inferred based on mobile phone signaling data The origin and destination intensity (OD intensity) of the region is used to reflect the regional commuting tidal pressure; Indicates road Level, such as main road, secondary road, expressway, etc.; Indicates road The physical length of Indicates time Meteorological data (such as rainfall, visibility, etc.); Indicates road At the moment Whether there are sudden traffic events (such as accidents, road closures, etc.). By introducing derived features constructed from GPS data from commercial vehicles and mobile phone signaling, this embodiment can fully supplement dynamic traffic perception information when fixed monitoring points are insufficiently covered or some roads are not equipped with sensors. This improves the model's ability to model actual urban traffic flow patterns and enhances overall prediction robustness.

[0166] Furthermore, in order to ensure that the input features have similar numerical ranges, the maximum-minimum normalization method can be used to normalize each type of feature data so that all data are distributed in the interval [0,1].

[0167] S150. Input the dynamic traffic network graph into the GCN model and the GAT model respectively; the GCN model extracts global spatial features based on the weight of each edge and the current feature representation of each node; the GAT model strengthens the role of key nodes to obtain local attention spatial features.

[0168] This step is based on the adaptive traffic network topology map constructed by S140. It uses a graph neural network to model the spatial relationship between different roads and extract the spatial feature representation of each node at the current moment as the input basis for the subsequent time series modeling module.

[0169] Alternatively, a combination of GCN (Graph Convolutional Network) and GAT (Graph Attention Network) can be used to enable the GNN model to capture global topological information while also enhancing its modeling capabilities for key nodes (such as intersections and congested bottlenecks). The GCN model can model the global topology, ensuring more stable spatial information propagation; the GAT model can enhance the influence of key road sections (such as intersections and highway ramps), making spatial feature extraction more targeted.

[0170] In one embodiment, the GCN model uses the adjacency matrix Perform information aggregation so that the features of each road node can absorb the information of neighboring roads:

[0171]

[0172] in, and Respectively represent the nodes in the GCN model Layer and The current feature representation of the layer (i.e., the feature representation at the current moment); represents the adjacency matrix with self-loops added, , represents the identity matrix of the same order; express degree matrix of ; is a learnable parameter matrix; Represents a nonlinear activation function, such as ReLU (Rectified Linear Unit).

[0173] The role of the GCN model is to smooth traffic status information and prevent drastic fluctuations in prediction results between adjacent roads; at the same time, it strengthens global topological information and improves the model's understanding of the network structure.

[0174] In the GAT model, the adjacency matrix is ​​used to limit the set of neighbors of a node (attention weights are only calculated for connected pairs of nodes) to prevent irrelevant nodes from spreading information. Because different roads have different influences on the target road, the GAT model introduces an attention mechanism to adaptively assign different importance to each neighbor:

[0175]

[0176]

[0177] in, and They represent the road i in the GAT model. Layer and The current feature representation of the layer, represents the set of neighbor nodes of road i, Indicates the GAT model Nodes in the layer The attention weight relative to node i, The GAT model The learnable feature map matrix of the layer, is the learnable vector of attention coefficients, for The transpose of Represents a vector or matrix concatenation operation (i.e., concatenating two vectors or matrices into a new vector or matrix in a specific dimension). represents a leaky linear rectifier function, which is used to introduce nonlinearity and avoid the gradient vanishing problem. and All are collections The road index in , and Represents roads and roads In the GAT model The current feature representation of the layer.

[0178] By introducing a dynamic and learnable attention weight mechanism, the GAT model can assign different influences to different neighbor nodes instead of simply averaging neighbor information. If the characteristics of a neighbor (such as an intersection or main road) change more significantly or become more important, it will be adaptively given a larger weight, thereby increasing the influence of key sections (such as intersections, main roads, etc.) on the overall prediction; and enhancing the modeling capabilities of complex traffic flow patterns (such as ramp merging and signal-controlled intersections).

[0179] Furthermore, in order to comprehensively utilize the global feature extraction capability of the GCN model and the local attention mechanism of the GAT model, this embodiment adopts a weighted fusion strategy to obtain the final spatial features:

[0180]

[0181] in, Indicates road At the moment Final spatial characteristics; Indicates road At the moment The global spatial features of , which are the outputs of the GCN model; Indicates road At the moment The local attention space features of , which are the outputs of the GAT model; and are all trainable weight parameters.

[0182] In summary, S140 models the spatial correlation between roads based on GNN, depicts the diffusion pattern of traffic flow and congestion in the road network, and integrates the characteristics of GCN and GAT, taking into account the global topological structure and key road section information.

[0183] S160 , performing short-term time series modeling and long-term time series modeling on the time series of the multimodal traffic data of each node before the time to be predicted, and obtaining short-term features and long-term features respectively.

[0184] This step models the temporal dynamics of traffic conditions to fully capture both short-term fluctuations and long-term trends. Optionally, an LSTM (Long Short Term Memory) model can be used for short-term dynamic forecasting to model short-term traffic fluctuations, such as congestion caused by traffic lights and accidents. A Transformer model can also be used for long-term trend learning to extract cyclical traffic patterns, such as morning and evening rush hours and weekend travel trends.

[0185] In a specific embodiment, the time series modeling process may include the following steps:

[0186] Step 1: Construct the spatial feature time series to be processed. At the moment Time series For the moment Before The spatial feature representation composition at each moment is expressed as follows:

[0187]

[0188] in, Indicates the The path obtained by fusion of GCN and GAT models at all times According to the needs of the modeling task, two spatial feature time series of different lengths can be constructed for each road at each moment, one for short-term prediction and the other for long-term modeling.

[0189] Step 2: Input the shorter time series into the LSTM model to perform short-term time series modeling. The LSTM model is better at capturing dependencies over longer time spans than the traditional RNN (Recurrent Neural Network). Specifically, for each road , the short-term time series features extracted by LSTM can be expressed as:

[0190]

[0191] in, Indicates road At the moment The short-term time series characteristics of Represents an operation in the LSTM model.

[0192] At the same time, a longer time series is input into the Transformer model for long-term time series modeling. Transformer needs to observe data over a longer time span to learn the periodic travel pattern rules, where the self-attention mechanism can efficiently model the global dependencies between different time steps. Specifically, for each road , the long-term temporal features extracted by Transformer can be expressed as:

[0193]

[0194] in, Indicates road At the moment The long-term temporal characteristics of Represents the operations in the Transformer model, mainly including:

[0195] Multi-head self-attention calculation:

[0196]

[0197] in, They are query, key and value matrices, respectively, all of which are input time series Obtained by linear transformation; Represents the similarity calculation between the query and the key; is the dimension of the key vector (used for scaling); Represents the normalized exponential function used to calculate the attention weight ; The final output of the model is the value matrix after weighted summation , used to represent the long-term time series characteristics of the current moment.

[0198] S170. Predict the traffic status at the time to be predicted based on the global spatial features, local attention spatial features, short-term features, and long-term features.

[0199] This step first performs spatiotemporal feature fusion, fusing the spatial features extracted by GNN, the short-term time series features extracted by LSTM, and the long-term time series features extracted by Transformer to form a complete spatiotemporal feature expression, thereby improving the ability to predict sudden events and periodic patterns.

[0200] Optionally, the fused features can be expressed as:

[0201]

[0202] in, Indicates road At the moment The spatiotemporal fusion characteristics of 、 and These are learnable fusion weights, whose values ​​are continuously adjusted during training to minimize the prediction error. The model automatically learns the relative importance of spatial features, short-term features, and long-term features in different scenarios.

[0203] After feature fusion, traffic status prediction and intelligent decision support can be performed based on spatiotemporal fusion features. Optionally, end-to-end learning can be performed through a multi-layer perceptron to output traffic flow, vehicle speed, and congestion index at future moments. This process can be expressed as:

[0204]

[0205] in, Indicates road At the moment Traffic conditions, including traffic flow, vehicle speed and congestion index, Represents an operation in a multilayer perceptron.

[0206] Furthermore, the above-mentioned dynamic traffic network graph, GCN model, GAT model, LSTM model and Transformer model together constitute a complete traffic status prediction model, and the parameters of each part of the model can be collaboratively determined through joint training. Optionally, a supervisory loss (such as mean square error, cross entropy, etc.) can be constructed between the model prediction result of the traffic status and the actual traffic status. The supervisory loss and the polynomial to be minimized in the above-mentioned least squares method together constitute a complete loss function, thereby completing the model training; the above-mentioned least squares method can also be separated from the model training, and the optimal distance attenuation coefficient is pre-determined by the least squares method, and then the model training is performed based on this coefficient. At this time, the loss function only includes the above-mentioned supervisory loss, and the distance attenuation coefficient is updated once every period of time according to the latest traffic flow, and the model is fine-tuned once. The adjustable parameters in the entire prediction model are jointly optimized through gradient backpropagation, and the parameter update follows the following formula:

[0207]

[0208] in, represents all learnable parameters in the entire prediction model, represents the updated parameters, represents the learning rate, is the loss function value.

[0209] Based on the trained model, predictions of future traffic conditions can be obtained, providing data support for signal optimization, route guidance, and public transportation resource allocation, thereby improving the intelligent level of traffic management. Specifically, in terms of signal optimization, signal cycles and phase times can be adjusted based on future traffic forecasts to improve road capacity. In terms of route guidance, the predicted congestion index can be used to provide optimized route recommendations for the navigation system, reducing the risk of traffic congestion. In terms of public transportation resource allocation, the predicted tidal effect of passenger flow can be combined to optimize the scheduling of buses, taxis, and online ride-hailing services, thereby improving transportation efficiency.

[0210] In summary, this embodiment provides a traffic state prediction method based on multimodal data and adaptive topology modeling. It integrates GPS data from commercial vehicles, mobile phone signaling data, fixed monitoring point data, meteorological data, and traffic event data. It optimizes data quality through methods such as anomaly detection, data noise reduction, and missing value filling, providing a sound data foundation for traffic state prediction. In traffic state prediction, this embodiment constructs an adaptive adjacency matrix based on road categories and historical traffic flows, dynamically adjusting the traffic network topology so that the influence weights between roads can be adjusted over time. This matrix is ​​then applied to a spatial feature extraction model, dynamically extracting global topological information from the road network using the GCN model and leveraging the multi-layer attention mechanism of the GAT model to strengthen the dependencies between key roads. Simultaneously, the optimized multimodal data time series are fed into LSTM and Transformer models, respectively, to model short-term emergencies and long-term travel patterns, further improving prediction accuracy. Finally, the global spatial features, local attention spatial features, short-term temporal features, and long-term temporal features are fused and fed into a multi-layer perceptron for regression calculation, outputting future traffic flow, vehicle speed, and congestion index. This embodiment effectively improves the accuracy and robustness of traffic status prediction through adaptive dynamic topology modeling, graph neural network optimization, and time series feature fusion. It can be applied to intelligent traffic management, signal optimization, road planning, and congestion control, providing more efficient data support.

[0211] It should be noted that the user data involved in this application (including operating vehicle GPS data, mobile phone signaling data, etc.) are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of relevant countries and regions, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0212] Figure 3 A schematic diagram of the structure of an electronic device provided by an embodiment of the present invention is shown in FIG. Figure 3As shown, the device includes a processor 60, a memory 61, an input device 62 and an output device 63; the number of processors 60 in the device can be one or more. Figure 3 In the embodiment, a processor 60 is used as an example; the processor 60, the memory 61, the input device 62 and the output device 63 in the device can be connected by a bus or other means. Figure 3 The bus connection is taken as an example.

[0213] Memory 61, as a computer-readable storage medium, can be used to store software programs, computer-executable programs, and modules, such as the program instructions / modules corresponding to the traffic state prediction method based on multimodal data and adaptive topology modeling in the embodiments of the present invention. Processor 60 executes the software programs, instructions, and modules stored in memory 61 to perform various functional applications and data processing of the device, thereby implementing the aforementioned traffic state prediction method based on multimodal data and adaptive topology modeling.

[0214] The memory 61 may primarily include a program storage area and a data storage area. The program storage area may store an operating system and at least one application required for a function; the data storage area may store data generated based on the use of the terminal. Furthermore, the memory 61 may include high-speed random access memory and non-volatile memory, such as at least one disk storage device, flash memory device, or other non-volatile solid-state memory device. In some instances, the memory 61 may further include memory remotely located relative to the processor 60, and these remote memories may be connected to the device via a network. Examples of such networks include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0215] The input device 62 may be used to receive input digital or character information and generate key signal input related to user settings and function control of the device. The output device 63 may include a display device such as a display screen.

[0216] An embodiment of the present invention further provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the traffic state prediction method based on multimodal data and adaptive topology modeling of any embodiment.

[0217] The computer storage medium of the embodiments of the present invention may adopt any combination of one or more computer-readable media. The computer-readable medium may be a computer-readable signal medium or a computer-readable storage medium. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or device, or any combination thereof. More specific examples (a non-exhaustive list) of computer-readable storage media include: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In this document, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, device, or device.

[0218] A computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, which carries computer-readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium that can transmit, propagate, or transport a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0219] Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

[0220] Computer program code for performing the operations of the present invention can be written in one or more programming languages, or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as C or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a stand-alone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computer (e.g., through the Internet using an Internet service provider).

[0221] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the above embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the technical solutions of the embodiments of the present invention.

Claims

1. A traffic state prediction method based on multimodal data and adaptive topology modeling, characterized in that: include: Obtain an initial traffic network diagram of the area to be predicted, wherein the traffic network diagram uses roads as nodes and the connections between roads as edges; Based on the type of each road and the historical traffic flow of each road in the recent period, the adjacency matrix between roads is dynamically determined, where each element in the adjacency matrix is ​​used to represent the degree of mutual influence between the traffic of two roads; In the initial traffic network graph, weights are assigned to each edge according to the adjacency matrix, and a current feature representation of each node is constructed according to the current multimodal traffic data of each road to obtain a dynamic traffic network graph; The dynamic traffic network graph is input into the GCN model and the GAT model respectively; the GCN model extracts global spatial features based on the weight of each edge and the current feature representation of each node; the GAT model strengthens the role of key nodes to obtain local attention spatial features; For the time series of multimodal traffic data of each node in the most recent period, short-term time series modeling and long-term time series modeling are performed respectively to obtain short-term features and long-term features; predicting future traffic conditions based on the global spatial features, the local attention spatial features, the short-term features, and the long-term features; The method of dynamically determining the adjacency matrix between roads based on the type of each road and the historical traffic flow of each road in the recent period includes: Based on the least squares method, the distance decay coefficient between each two adjacent roads to maintain consistent traffic flow is estimated : ， in, Indicates road i and roads j There are two adjacent roads. and Represents roads i and roads j In the recent period of historical traffic flow, Indicates road i and roads j geographical distance, represents the exponential function with the natural constant e as the base, Indicates that it can Take the minimum value value; According to the following formula, the degree of mutual influence of traffic between each two roads is dynamically determined : , in, and Represents roads i and roads j Type, Indicates the distance attenuation coefficient between adjacent roads of the same type, Indicates the distance attenuation coefficient between adjacent roads of different types, represents a learnable cross-road type factor.

2. The method according to claim 1, characterized in that The GCN model extracts global spatial features based on the weight of each edge and the current feature representation of each node, including: The global spatial features are extracted by the GCN model according to the following formula: ， in, and Respectively represent the roads in the GCN model Layer and The current feature representation of the layer; represents the adjacency matrix with self-loops added, , represents the adjacency matrix, represents the identity matrix of the same order; express degree matrix of ; represents the learnable parameter matrix; represents a non-linear activation function.

3. The method according to claim 1, characterized in that The GAT model strengthens the role of key nodes to obtain local attention space features, including: Determine the neighbor nodes of each node based on the weight of each edge; The GAT model strengthens the role of key nodes according to the following formula to obtain the local attention space feature: ， ， in, and Represents roads i In the GAT model Layer and The current feature representation of the layer, represents a nonlinear activation function, Indicates road i The set of neighbor nodes of Indicates the GAT model Road in the layer j Relative to the road i The attention weight, The GAT model The learnable feature map matrix of the layer, is the learnable vector of attention coefficients, for The transpose of Represents the concatenation operation of vectors or matrices, represents the linear rectification function with leakage, represents the exponential function with the natural constant e as the base, and j Both represent sets The road index in , and Represents roads k and roads j In the GAT model The current feature representation of the layer.

4. The method according to claim 1, wherein The obtaining of an initial traffic network diagram of the area to be predicted includes: The connection relationship between two roads with the same direction and physically connected is determined as a direct connection; The connection relationship between two intersection roads with different directions is determined as an intersection connection; The connection relationship between the expressway and its on- and off-ramps is determined as the expressway ramp connection.

5. The method according to claim 1, wherein The constructing of the current feature representation of each node based on the current multimodal traffic data of each road includes: Obtain GPS data of current operating vehicles in the forecasted area, mobile phone signaling data, traffic flow data at fixed monitoring points, meteorological data, and traffic event data; Determine the current traffic flow, vehicle speed, road grade, weather characteristics, and traffic event characteristics for each road in the area based on the acquired data; Based on the current data and road level determined for each road, the current feature representation of each node is constructed.

6. An electronic device, characterized in that: include: one or more processors; a memory for storing one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors implement the traffic status prediction method based on multimodal data and adaptive topology modeling as described in any one of claims 1-5.

7. A computer-readable storage medium, characterized in that A computer program is stored thereon, which, when executed by a processor, implements the traffic state prediction method based on multimodal data and adaptive topology modeling as described in any one of claims 1-5.

Images