A smart traffic congestion governance method and system based on multi-modal data fusion

Abstract

The application relates to the technical field of intelligent traffic, and provides an intelligent traffic congestion governance method and system based on multi-modal data fusion, which comprises the following steps: preprocessing basic traffic data and associated traffic data; performing multi-modal fusion on the pretreated basic traffic data and associated traffic data based on a data fusion unit to obtain traffic state fusion data; obtaining road network space data and historical congestion time series data, and constructing a traffic state space-time feature map according to the road network space data and the historical congestion time series data; taking the traffic state fusion data as the input of a congestion prediction model, dynamically capturing congestion key influence factors of different road sections and different time periods through an attention mechanism, and obtaining a congestion prediction result; obtaining multiple groups of candidate traffic governance strategies according to the congestion prediction result, and obtaining an optimal governance strategy according to the multiple groups of candidate traffic governance strategies. The application can improve the accuracy of congestion prediction.

Description

Technical Field

[0001] This invention relates to the field of intelligent transportation technology, and more specifically, to an intelligent traffic congestion management method and system based on multimodal data fusion. Background Technology

[0002] With the acceleration of urbanization, urban public transportation congestion is becoming increasingly severe. Existing technologies mainly rely on single applications of the Internet of Things (IoT) and Artificial Intelligence (AI), such as collecting traffic flow data through sensors or using machine learning models for traffic condition prediction. However, current solutions have limitations in terms of data collection dimensions, relying solely on traditional sensors (such as cameras and GPS) and failing to integrate multi-source data such as social media, weather, and road construction, resulting in incomplete input information for the models.

[0003] Therefore, existing intelligent traffic congestion management systems are prone to low accuracy in congestion prediction and failure of management strategies because they cannot integrate multi-source heterogeneous data in real time.

[0004] Therefore, it is necessary to develop a smart traffic congestion management method based on multimodal data fusion and dynamic decision-making to improve the accuracy of congestion prediction. Summary of the Invention

[0005] To improve the accuracy of congestion prediction, this invention provides a smart traffic collaborative management method and system based on large models and federated learning, the specific technical solution of which is as follows:

[0006] A smart traffic congestion management method based on multimodal data fusion includes the following steps: Acquire basic traffic data and related traffic data from multiple different dimensions, and preprocess the basic traffic data and related traffic data; A data fusion unit is constructed, and based on the data fusion unit, multimodal fusion is performed on the preprocessed basic traffic data and related traffic data to obtain traffic status fusion data; Obtain road network spatial data and historical congestion time series data, and construct a spatiotemporal feature map of traffic status based on the road network spatial data and historical congestion time series data; A congestion prediction model is constructed, using traffic state fusion data as input. The model dynamically captures key congestion influencing factors in different road sections and time periods through an attention mechanism to obtain congestion prediction results. Multiple candidate traffic management strategies are obtained based on congestion prediction results, and the optimal management strategy is obtained based on these multiple candidate strategies.

[0007] The intelligent traffic congestion management method based on multimodal data fusion described in this invention acquires and preprocesses basic and related traffic data from multiple dimensions, enabling the full-dimensional collection and fusion of multi-source heterogeneous data. This significantly improves the quality of traffic data and the data foundation for congestion prediction. Furthermore, by dynamically capturing key congestion influencing factors in different road segments and time periods through an attention mechanism, congestion prediction results are obtained, which helps improve the accuracy of congestion prediction and achieve real-time, accurate traffic congestion prediction.

[0008] In addition, by obtaining multiple candidate traffic management strategies based on congestion prediction results and then selecting the optimal management strategy from these multiple candidate strategies, dynamic joint optimization of traffic management strategies can be achieved, significantly improving the efficiency and accuracy of congestion management and avoiding the implementation of ineffective management strategies.

[0009] In summary, the proposed intelligent traffic congestion management method addresses the problem that existing technologies, due to their inability to integrate multi-source heterogeneous data in real time, often result in low accuracy in congestion prediction and ineffective management strategies, thereby improving the accuracy of congestion prediction.

[0010] Preferably, the intelligent traffic congestion management method further includes the following steps: The optimal governance strategy is published on the blockchain to achieve full-process traceability of shared data among multiple departments during the implementation of the governance strategy; Multi-departmental collaboration is essential to implement optimal governance strategies.

[0011] Preferably, the intelligent traffic congestion management method further includes the following steps: It receives real-time traffic congestion information reported by users and provides personalized route recommendations based on real-time traffic congestion information, basic traffic data, and related traffic data. By incentivizing users to participate in data collection through a reward mechanism, the collected user data is preprocessed and used as supplementary data for model training, thereby enabling iterative optimization of congestion prediction models and governance strategies.

[0012] Preferably, the intelligent traffic congestion management method further includes the following steps: For different new traffic scenarios, the congestion prediction model is adapted based on transfer learning; The parameters of the congestion prediction model are iteratively updated at preset time intervals.

[0013] Preferably, the specific method for obtaining the optimal traffic management strategy based on multiple candidate traffic management strategies includes the following steps: Construct a digital twin virtual traffic environment and input multiple sets of candidate traffic management strategies into the digital twin virtual traffic environment to conduct effect simulation; The optimal governance strategy is selected based on the results of the preliminary simulation.

[0014] A smart traffic congestion management system based on multimodal data fusion, used to implement the aforementioned smart traffic congestion management method based on multimodal data fusion, includes: The data acquisition and fusion module is used to acquire basic traffic data and related traffic data from multiple different dimensions, preprocess the basic traffic data and related traffic data, perform multimodal fusion on the preprocessed basic traffic data and related traffic data, acquire traffic state fusion data, acquire road network spatial data and historical congestion time series data, and construct a traffic state spatiotemporal feature map based on the road network spatial data and historical congestion time series data. The congestion prediction module is used to dynamically capture key congestion influencing factors in different road sections and time periods through attention mechanism based on traffic condition fusion data to obtain congestion prediction results. The governance decision module is used to obtain multiple sets of candidate traffic governance strategies based on congestion prediction results, and to obtain the optimal governance strategy based on the multiple sets of candidate traffic governance strategies.

[0015] Preferably, the intelligent traffic congestion management system further includes: The cross-departmental collaborative governance module is used to publish the optimal governance strategy on the blockchain, so as to achieve full-process traceability of shared data among multiple departments during the execution of the governance strategy, and to enable multi-departmental collaboration to execute the optimal governance strategy.

[0016] Preferably, the intelligent traffic congestion management system further includes: The user feedback interaction module is used to receive real-time congestion information reported by users. Based on the real-time congestion information, basic traffic data, and related traffic data, it provides users with personalized route recommendations and incentivizes users to participate in data collection through a reward mechanism. The collected user data is preprocessed and used as supplementary data for model training to achieve iterative optimization of the congestion prediction model and governance strategies.

[0017] Preferably, the governance decision-making module is further used for: For different new traffic scenarios, the congestion prediction model is adapted based on transfer learning, and the parameters of the congestion prediction model are iteratively updated at preset time intervals.

[0018] Preferably, the governance decision-making module is further used for: A digital twin virtual traffic environment is constructed, and multiple sets of candidate traffic management strategies are input into the digital twin virtual traffic environment to conduct effect simulations. The optimal management strategy is then selected based on the simulation results. Attached Figure Description

[0019] The invention will be further understood from the following description taken in conjunction with the accompanying drawings. The components in the drawings are not necessarily drawn to scale, but rather the emphasis is on illustrating the principles of the embodiments. In different views, the same reference numerals designate corresponding parts.

[0020] Figure 1 This is a schematic diagram of the overall process of a smart traffic congestion management method based on multimodal data fusion in one embodiment of the present invention; Figure 2 This is a flowchart illustrating a specific method for obtaining the optimal traffic management strategy based on multiple candidate traffic management strategies in one embodiment of the present invention. Figure 3 This is one of the overall process diagrams of a smart traffic congestion management method based on multimodal data fusion in another embodiment of the present invention; Figure 4 This is the second schematic diagram of the overall process of a smart traffic congestion management method based on multimodal data fusion in another embodiment of the present invention. Detailed Implementation

[0021] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to its embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and do not limit the scope of protection of the invention.

[0022] It should be noted that when an element is referred to as being "fixed to" another element, it can be directly attached to the other element or there may be an intervening element. When an element is referred to as being "connected to" another element, it can be directly connected to the other element or there may be an intervening element. The terms "vertical," "horizontal," "left," "right," and similar expressions used herein are for illustrative purposes only and do not represent the only possible implementation.

[0023] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. The terminology used herein in the description of the invention is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. The term "and / or" as used herein includes any and all combinations of one or more of the associated listed items.

[0024] In this invention, "first" and "second" do not represent a specific quantity or order, but are merely used to distinguish names.

[0025] Before describing the embodiments of the present invention, a brief introduction to the prior art will be given.

[0026] With the rapid advancement of urbanization, the urban population and the number of motor vehicles continue to grow. Urban road traffic congestion has become one of the core pain points restricting urban development. It not only reduces residents' travel efficiency and increases travel costs, but also easily leads to traffic accidents and exacerbates urban environmental pollution. Therefore, efficient intelligent traffic congestion management technology has become an urgent need for urban transportation development. At present, the industry has gradually applied Internet of Things (IoT) and Artificial Intelligence (AI) technologies to the field of traffic congestion management. Existing technical solutions mainly collect traffic flow data by deploying traditional sensors such as cameras and GPS, and then combine them with machine learning models to predict traffic conditions. At the same time, they rely on manually preset rules to realize basic management operations such as traffic signal control and public transportation scheduling, which alleviates local traffic congestion problems to a certain extent.

[0027] However, existing intelligent traffic congestion management technologies still have many intractable technical shortcomings, failing to meet the congestion management needs in complex urban traffic scenarios, specifically: First, there are obvious shortcomings in data collection and fusion. Existing technologies rely solely on traditional sensing devices to collect single traffic data, without integrating multi-source heterogeneous data such as social media, weather, road construction, and traffic incidents. Furthermore, there is a lack of efficient multi-source data fusion methods. At the same time, there is a serious risk of privacy leakage during the data sharing process among multiple departments, resulting in incomplete input information for congestion prediction models and difficulty in guaranteeing data quality.

[0028] Secondly, the adaptability and generalization ability of congestion prediction models are weak. Most existing prediction models are static models, which cannot dynamically capture the differences in traffic patterns on different road sections and at different times. They are also difficult to adapt quickly to special traffic scenarios such as holidays and extreme weather. Furthermore, the model parameters are updated late, and the ability to respond to sudden traffic events such as traffic accidents is insufficient. Ultimately, this results in low accuracy of congestion prediction, with prediction errors exceeding 20% ​​during peak hours.

[0029] Third, traffic congestion management strategies lack dynamic optimization capabilities. Existing technologies such as traffic signal timing and bus departure interval scheduling rely on fixed rules preset by humans, failing to achieve joint optimization of traffic signal control and bus scheduling. Furthermore, there is no practical effect verification process before the management strategy is formulated, which easily leads to a mismatch between the management strategy and the actual traffic conditions, resulting in a significant reduction in the management effect.

[0030] Fourth, the lack of a multi-departmental collaborative governance mechanism, the absence of an efficient information exchange and collaborative management platform among relevant entities such as traffic police, bus companies, and meteorological departments, low data sharing efficiency, and difficulty in synchronous cooperation among departments during the implementation of governance strategies further reduce the overall efficiency of congestion management.

[0031] Fifth, existing technologies have not built a user-participatory traffic management system, relying solely on equipment and departmental data collection. The coverage and real-time nature of data collection are limited, making it impossible to capture sudden congestion in local road sections in a timely manner.

[0032] In summary, the core problem of existing intelligent traffic congestion management technologies is the inability to achieve real-time fusion and privacy protection of multi-source heterogeneous data, which leads to low accuracy in congestion prediction and ineffective management strategies. It is also accompanied by a series of problems such as poor model adaptability, static management strategies, and insufficient cross-departmental collaboration capabilities. Therefore, there is an urgent need to develop an intelligent traffic congestion management technology that can achieve multi-modal data fusion, dynamic and accurate prediction, intelligent decision-making and governance, and cross-departmental collaborative management.

[0033] One of the objectives of this invention is to improve the accuracy of congestion prediction. To this end, such as... Figure 1 As shown, a smart traffic congestion management method based on multimodal data fusion includes the following steps: S1 acquires basic traffic data and related traffic data from multiple different dimensions, and preprocesses the basic traffic data and related traffic data.

[0034] Specifically, basic traffic data, including traffic flow and vehicle trajectories, can be acquired through sensors such as LiDAR and millimeter-wave radar deployed at traffic nodes, as well as vehicle-mounted terminals. Related traffic data, such as weather and road event data, can be obtained through social media API interfaces. Preprocessing of the basic and related traffic data includes, but is not limited to, cleaning, deduplication, and normalization. After preprocessing, the valid data is uploaded to the cloud.

[0035] S2, construct a data fusion unit, and perform multimodal fusion on the preprocessed basic traffic data and related traffic data based on the data fusion unit to obtain traffic status fusion data.

[0036] Specifically, cloud-based GNN fusion units can be used as data fusion units to fuse preprocessed multimodal data and obtain traffic status fusion data.

[0037] S3: Obtain road network spatial data and historical congestion time series data, and construct a spatiotemporal feature map of traffic status based on the road network spatial data and historical congestion time series data.

[0038] By combining road network spatial data and historical congestion time series data, a spatiotemporal feature map of traffic status is constructed. At the same time, through federated learning privacy aggregation units, heterogeneous data from multiple departments such as traffic police, bus companies, and meteorological departments are aggregated to supplement traffic status feature information while protecting the data privacy of each department.

[0039] As a preferred technical solution, the specific method for constructing a spatiotemporal feature map of traffic conditions includes the following steps: The first step involves secondary normalization and feature standardization of multi-source data. The data fusion unit receives preprocessed multimodal data (i.e., basic traffic data and related traffic data), locally stored road network spatial basic data (road network topology, road segment numbers, intersection / bus stop coordinates, number of lanes, speed limits, etc.), and historical congestion time series data (congestion index, traffic speed, average vehicle delay, etc. for different time periods and road segments in the past 1 / 3 months). Secondary normalization and standardization are performed on these three types of data. For example, feature dimension alignment is performed on real-time multimodal data to eliminate dimensional differences; topological quantization is performed on road network spatial data; a unique spatial identifier ID is assigned to each traffic node (intersection, bus stop, road segment start and end point); an index of connectivity relationships between nodes is established; and time dimension alignment is performed on historical congestion time series data, divided into 5-minute / 10-minute time steps, extracting core congestion features such as congestion index and traffic speed for each time step, and interpolating missing values ​​to complete the normalization.

[0040] The second step is to construct the basic undirected / directed graph structure G=(V,E,A) of the transportation network. Based on the spatial topology of the urban transportation network, the basic graph structure of the GNN is constructed, and the core elements of the graph are defined as nodes V, edges E, and adjacency matrix A.

[0041] Key traffic nodes in the urban road network are used as nodes in a graph, including intersections, bus stops, and intermediate monitoring points along road segments. Each node corresponds to a unique spatial identifier (ID), and the initial attributes of the nodes are the real-time multimodal feature vectors normalized in step 1. The connectivity between nodes is represented as edges in the graph. If there is an actual road segment between two traffic nodes (e.g., from one intersection to an adjacent intersection, or from one intersection to a bus stop), an edge is established. The edge attributes consist of the basic features of the corresponding road segment plus real-time traffic features (segment length, number of lanes, speed limit, real-time traffic speed, real-time average vehicle delay, etc.). An adjacency matrix is ​​constructed based on the connectivity between nodes. If there is an edge connecting node i and node j, then Aij = 1; otherwise, Aij = 0. If the traffic direction of a road segment is considered (e.g., one-way streets), a directed adjacency matrix is ​​constructed, and Aij is assigned according to the traffic direction. Simultaneously, the adjacency matrix is ​​normalized (e.g., row normalization) to avoid gradient explosion during GNN training.

[0042] The third step is the extraction and spatiotemporal embedding of historical congestion time series features. The temporal dimension features of the historical congestion time series data are extracted and embedded into the basic graph structure constructed in the second step, achieving a preliminary correlation between spatial and temporal features.

[0043] The fourth step is spatiotemporal feature aggregation and neighborhood propagation based on GNN. A Graph Attention Network (GAT) / Graph Convolutional Network (GCN) is used as the core GNN model to perform spatial neighborhood aggregation and deep spatiotemporal feature fusion of the attribute features of each node in the basic graph structure. This enables the propagation and correlation of traffic state features in the road network. Specifically, for each node in the graph structure, the features of its k-order neighboring nodes are aggregated through the convolutional / attention layers of the GNN to capture the spatial correlation of traffic states in the road network (e.g., the feature that congestion at one intersection will spread to adjacent intersections); a spatiotemporal attention mechanism is introduced to assign weights to the aggregated features; and through the stacking of multiple GNN networks, the global spatiotemporal features of the traffic network are extracted, resulting in a fused spatiotemporal feature vector for each node. This vector contains both the node's own real-time multimodal features and historical congestion time features, as well as the correlation features of neighboring nodes, comprehensively representing the traffic state of the node.

[0044] The fifth step is the structured generation of the traffic state spatiotemporal feature map. Based on the geospatial coordinates of the urban traffic network, the spatiotemporal feature vectors of the nodes and edge features fused by GNN in the fourth step are structured and visualized / numerically generated to obtain the traffic state spatiotemporal feature map. This feature map is a dual-map form of numerical structured map + geographic visualization map, which takes into account both the practicality of subsequent model input and the intuitiveness of manual monitoring.

[0045] Step 6: Dynamic updating and anomaly removal optimization of the spatiotemporal feature map. To adapt to the real-time dynamic changes in urban traffic, the generated traffic state spatiotemporal feature map is dynamically updated and outliers are removed to ensure the accuracy and timeliness of the feature map.

[0046] Step 7: Standardized output of the spatiotemporal feature map. The optimized traffic state spatiotemporal feature map is output to the adaptive dynamic congestion prediction module in a standardized data format (such as tensors and matrices) as the core input data of the Transformer prediction model. At the same time, the visualized spatiotemporal feature map is synchronized to the blockchain cross-departmental collaborative governance platform to provide traffic police, bus companies and other departments with intuitive traffic status references and realize multi-module and multi-departmental data sharing.

[0047] S4. Construct a congestion prediction model. Use traffic state fusion data as input to the congestion prediction model. Dynamically capture key congestion influencing factors in different road segments and time periods through an attention mechanism to obtain congestion prediction results.

[0048] Specifically, the road segment node features and time-series features of the traffic state spatiotemporal feature map are first vectorized and encoded into feature tensors recognizable by the Transformer model, preserving basic information such as road segment topology and time-series congestion trends. Spatial attention sublayers and temporal attention sublayers are then built in the Transformer, and feature attention weights are calculated for road segments and time periods, respectively. Then, the traffic correlation between each road segment (such as the congestion transmission between adjacent road segments) is calculated in the spatial dimension, and high weights are assigned to key influencing road segments. In the temporal dimension, the importance of congestion features in different time periods (off-peak / peak / sudden periods) is calculated, and high weights are assigned to core time periods, while low weights are given to features with no significant impact. Based on the calculated spatiotemporal attention weights, the high-weighted road segment and time period features are weighted and fused to select and strengthen key congestion influencing factor features (such as morning peak traffic volume on a certain road segment and road capacity under extreme weather conditions). Finally, the fused key congestion influencing factor feature tensor is output as the core input for subsequent congestion state prediction of the prediction model.

[0049] Preferably, the intelligent traffic congestion management method further includes the following steps: for different new traffic scenarios, the congestion prediction model is rapidly adapted based on transfer learning; the parameters of the congestion prediction model are iteratively updated at a preset time interval, such as every five minutes, to adapt to real-time traffic changes; and the congestion prediction results are processed by a Bayesian uncertainty quantification unit to output congestion prediction results containing confidence intervals.

[0050] Specifically, for new traffic scenarios such as new cities, holidays / extreme weather, etc., the model is based on the pre-trained model of the source scenario and is quickly adapted through small sample fine-tuning, with an adaptation time of ≤2 hours. First, a Transformer congestion prediction base model is trained using massive multimodal traffic data from the source city / regular traffic scenarios. The underlying general spatiotemporal feature extractor of the model is fixed (extracting common patterns of traffic congestion, such as the correlation between traffic flow and congestion index, and time-of-day travel characteristics). Then, a small amount of multimodal traffic data (small sample) from new city / holiday / extreme weather scenarios is collected. After edge computing preprocessing and GNN fusion, an adapted spatiotemporal feature map is generated, and the underlying general layer parameters of the pre-trained model are frozen. Only the top-level scenario adaptation layer of the model is fine-tuned, and a small number of parameters are updated for the feature differences of the new scenario (such as the new city road network structure, holiday travel patterns, and extreme weather road characteristics). Finally, the small sample features of the new scenario are input into the model for rapid iterative training to optimize the parameters of the top-level adaptation layer so that the model adapts to the congestion patterns of the new scenario. The model prediction effect is verified using validation set data of the new scenario. If the accuracy does not meet the standard, small parameter adjustments are made. After successful verification, the adapted model is deployed directly to complete the rapid adaptation to the new scenario.

[0051] S5: Obtain multiple candidate traffic management strategies based on congestion prediction results, and then obtain the optimal management strategy based on these multiple candidate strategies.

[0052] Specifically, the joint optimization objective, state space, and action space are first defined. The core of the multi-objective optimization is to minimize peak congestion duration, maximize bus punctuality, and optimize road network traffic efficiency. The state space incorporates traffic state spatiotemporal feature map data, real-time road network congestion distribution, current signal timing parameters, bus operation status, and constraints such as weather / road events. The action space is defined as adjustable parameters for traffic signal timing (signal cycle, green light ratio of each phase, phase difference) and adjustable parameters for bus departure intervals (departure intervals for different routes / time periods, station scheduling duration).

[0053] For example, the state space S is represented as .in, These are represented in sequence as congestion probability (i.e., the congestion prediction result of step S4), real-time traffic speed / density vector, bus GPS location, and scene feature vector. Let A represent the state at the current time t. The action space A is represented as... , These are represented, in order, the action at current time t, the signal timing vector, and the bus scheduling vector. Signal timing vector Used to control the green light phase duration ratio at k core intersections within a control area; the bus dispatch vector can be understood as the departure interval adjustment coefficient for M bus routes. and Between -0.5 and 0.5. When When the value is greater than zero, increase the frequency of bus services. If the actual green light duration is less than zero, increase the frequency of bus services; otherwise, maintain the baseline. For example, the actual green light duration = ×(Maximum green light duration - Minimum green light duration) + Minimum green light duration. Actual bus departure interval = Base interval × .

[0054] The scene feature vector consists of several parameters: weather, event, time period, and congestion level. After normalization, weather features can be set to 0 for sunny days, 1 for extreme weather, and 0.3 and 0.7 for light rain and heavy rain, respectively. Event feature = event impact range × event level / total road network area. Time period feature = Sigmoid(|current time - peak hour center time| / 3). Congestion feature can be set as the overall congestion index, representing the proportion of severely congested road sections within the area.

[0055] Preferably, a learnable mask matrix Mscene is introduced to dynamically mask inapplicable actions, resulting in the updated effective actions. Equal to the current action ⊙Mscene. It can take scene features as input and obtain a mask matrix based on multilayer perception.

[0056] Current action ⊙Mscene here uses element-wise multiplication. For each element in the action vector, it is multiplied by the corresponding mask value. When Mscene is zero, the action is completely masked; when Mscene is 1, the action is completely preserved; intermediate values ​​are scaled proportionally. In the PPO algorithm, the action a_t is a continuous vector obtained by sampling the distribution output of the policy network. The mask is applied to the sampled action. Due to the poor road conditions during heavy rain, increased bus frequency may lead to vehicle backlog and accidents, so it is generally necessary to limit the frequency of bus dispatching. Thus, in heavy rain, by masking the increased bus departure action, vehicle backlog can be prevented.

[0057] Next, a reward function for the PPO algorithm is constructed based on the optimization objectives. Positive rewards are given to decision-making behaviors that reduce congestion duration, improve punctuality, and improve road network efficiency, while negative rewards are given to decision-making behaviors that exacerbate congestion and cause public transport delays. The reward weights of multiple objectives are balanced through a weighted approach to adapt to actual traffic management needs.

[0058] For example, the multi-objective reward function is a weighted sum of traffic efficiency rewards, fairness rewards, and sustainability rewards.

[0059] Traffic efficiency reward is represented as .in, These are represented, in order, the total number of road segments in the target area, the actual speed of road segment i, the predicted speed, and the free-flow speed. This represents the confidence weighting term quantified by Bayesian uncertainty (1 - scaling factor × Var( This is used to avoid over-penalizing low-quality prediction results and to incentivize models to improve their prediction capabilities for road sections with high uncertainty. Var( represents the congestion probability of road segment i) ) represents the variance of the Bayesian uncertainty quantization output. It represents the speed deviation ratio. The square operation is used to amplify severe prediction errors and emphasizes the penalty for outliers.

[0060] This traffic efficiency reward is mainly used to improve the match between predicted and actual vehicle speeds, thereby optimizing road network traffic speeds and reducing congestion delays.

[0061] Fairness of reward .in, The Gini coefficient is represented by J, and the total number of urban governance zones is represented by J. These represent the delay time (i.e., the average travel delay per resident of community j) and the permanent population of community j, respectively. This represents population-weighted delay time, used to eliminate population size differences and reflect the delay burden borne by a unit of population. When the delay level of community j is higher than the regional median, improving its transportation services yields a higher reward. In general, the fairness reward function mainly avoids misjudging commercial areas (with low resident population) by correcting commuting dependence and filtering extreme scenarios to avoid regional discrimination in traffic management.

[0062] Sustainability Awards .in, These represent the normalization coefficient, the total number of bus routes, the total number of motor vehicles, and the adjustment amount of departure interval, respectively. These represent the real-time passenger load and departure stability weight for route m, respectively. The departure stability weight is mainly used to encourage stable departures and avoid passenger waiting uncertainty caused by frequent scheduling. This represents the number of vehicles equivalent in lane i. Generally, 1 bus can be converted to 2.5 private cars. The normalization coefficient is used to eliminate differences in city size.

[0063] In summary, the multi-objective reward function is defined as a weighted sum of traffic efficiency reward, fairness reward, and sustainability reward. It aims to achieve a dynamic balance between traffic efficiency, social equity, and green sustainability in traffic control systems. By using reinforcement learning to drive the agent to generate collaborative optimization strategies, it breaks through the limitations of traditional single-objective optimization and, for the first time, incorporates socio-economic indicators such as the Gini coefficient and carbon emission intensity into the real-time control loop.

[0064] This sustainability award has a dual incentive mechanism, designed to increase the attractiveness of public transport and curb private car dependence, thereby increasing public transport share and reducing carbon intensity.

[0065] A dual-decision network for the PPO algorithm, integrating traffic signal control and bus dispatching modules, is constructed. This network includes a policy network (for outputting decision actions) and a value network (for evaluating the value of actions). The model network parameters are initialized based on historical traffic data and road network topology information. The stability of model training is ensured through PPO's unique policy pruning mechanism.

[0066] Finally, the real-time traffic state spatiotemporal feature map data is input into the PPO joint optimization model, and the model is iteratively trained and the parameters are updated using online learning. For the current actual traffic state, the trained model outputs multiple sets of candidate governance strategies with different parameter combinations (each strategy includes a matching signal timing scheme + bus departure interval scheme), covering decision combinations with different optimization tendencies, and meeting the needs of pre-simulation comparison.

[0067] As a preferred technical solution, such as Figure 2As shown, the specific method for obtaining the optimal traffic management strategy based on multiple candidate traffic management strategies includes the following steps: S51. Construct a digital twin virtual traffic environment and input multiple sets of candidate traffic management strategies into the digital twin virtual traffic environment to conduct effect simulation.

[0068] S52, Select the optimal governance strategy based on the simulation results.

[0069] Specifically, the real-world urban road network topology, real-time traffic conditions (traffic flow, vehicle trajectories, congestion points), weather / road events, and other actual scenario information are accurately mapped 1:1 to the digital twin virtual traffic environment. At the same time, the signal timing and bus departure interval parameters of each candidate strategy are imported into the virtual environment to complete the initialization of the pre-rehearsal scenario corresponding to each strategy, ensuring the consistency between the virtual environment and the real traffic scenario.

[0070] In a digital twin environment, multiple candidate strategies are simulated in parallel or sequentially, with simulation durations set to match actual conditions (e.g., 30 / 60 minutes during peak hours). During the simulation, core evaluation indicators for each strategy are collected and quantified in real time, including average congestion duration during peak hours, bus punctuality rate, average traffic speed of the road network, and queue length at intersections.

[0071] Based on pre-defined joint optimization objectives, weighted averages are assigned to each core evaluation indicator to establish a multi-dimensional quantitative evaluation system. Using indicator data collected during the pre-simulation, each candidate strategy is comprehensively scored to quantify its governance effectiveness. Finally, based on the comprehensive scoring results, candidate strategies with poor governance effects or those that do not conform to actual traffic constraints are eliminated, and the strategy with the highest comprehensive evaluation score is selected as the optimal governance strategy. Simultaneously, 2-3 suboptimal strategies are retained as backup strategies to address unexpected changes in real-world traffic scenarios. Finally, the optimal governance strategy is standardized and output for practical implementation in traffic management.

[0072] By acquiring and preprocessing basic and related traffic data from multiple dimensions, this approach enables the comprehensive collection and fusion of multi-source heterogeneous data, significantly improving traffic data quality and the data foundation for congestion prediction. Furthermore, by dynamically capturing key congestion influencing factors across different road segments and time periods using an attention mechanism, congestion prediction results can be obtained, thereby improving prediction accuracy and achieving real-time, precise traffic congestion prediction.

[0073] In addition, by obtaining multiple candidate traffic management strategies based on congestion prediction results and then selecting the optimal management strategy from these multiple candidate strategies, dynamic joint optimization of traffic management strategies can be achieved, significantly improving the efficiency and accuracy of congestion management and avoiding the implementation of ineffective management strategies.

[0074] In summary, the proposed intelligent traffic congestion management method addresses the problem that existing technologies, due to their inability to integrate multi-source heterogeneous data in real time, often result in low accuracy in congestion prediction and ineffective management strategies, thereby improving the accuracy of congestion prediction.

[0075] In one embodiment, such as Figure 3 As shown, the intelligent traffic congestion management method further includes the following steps: S6 publishes the optimal governance strategy on the blockchain to achieve full-process traceability of shared data among multiple departments during the implementation of the governance strategy.

[0076] S7 involves multi-departmental collaboration to implement optimal governance strategies.

[0077] Specifically, the selected optimal governance strategy will be released through the blockchain cross-departmental collaborative governance module. The blockchain data traceability unit will enable full-process traceability of shared data among multiple departments during the execution of the governance strategy. At the same time, the cross-departmental real-time interaction unit will enable the coordinated cooperation of multiple departments such as traffic police and bus companies to execute the optimal governance strategy.

[0078] In one embodiment, such as Figure 4 As shown, the intelligent traffic congestion management method further includes the following steps: The S8 receives real-time traffic congestion information reported by users and provides personalized route recommendations based on real-time congestion information, basic traffic data, and related traffic data.

[0079] Generally, real-time congestion information can be reported and basic verification can be performed by the APP to ensure the integrity of the reported information. At the same time, the APP can obtain the user's real-time location and bind it to the reported data to complete the basic data collection. The APP will encrypt and transmit the verified congestion report data to the edge computing unit of the nearest traffic node. After rapid cleaning and preprocessing at the edge, it will be uploaded to the cloud multimodal data fusion module. It will be fused with real-time traffic flow, weather, road event and other data collected by the system to update the spatiotemporal feature map of traffic status in real time and refresh the congestion status of the entire road network simultaneously.

[0080] Then, the cloud integrates the updated spatiotemporal feature map of traffic conditions (real-time road network congestion distribution, traffic speed), historical traffic data collected by the system, real-time operation data of public transportation / road network, and constraint data such as weather / road construction, providing a comprehensive and real-time data source for route recommendation. The APP obtains the user's basic route planning requirements (starting point, destination, mode of transportation, such as driving / public transportation), while identifying the user's personalized preferences (such as fastest arrival, less congestion, public transportation priority, fewer transfers); the cloud calls the path planning algorithm (such as improved A... The algorithm (Dijkstra's algorithm) combines real-time traffic data to calculate multiple candidate routes and outputs core information such as the estimated travel time, congestion probability, and traffic status of key nodes for each route.

[0081] The cloud pushes the calculated candidate routes and related information to the app. The app displays recommended routes in order of user preference priority, while also marking real-time congestion points, construction / accident alerts, and real-time bus arrival information for each route. Users can select and start navigation with one click. During the user's journey, the app synchronizes traffic status data updated in the cloud in real time. If a sudden congestion occurs on the recommended route, the cloud quickly recalculates the optimal route, and the app promptly pushes route adjustment suggestions to the user, achieving dynamic adaptation of navigation routes. At the same time, the user's driving trajectory and route selection behavior are fed back to the system as supplementary data for model training.

[0082] S9 incentivizes users to participate in data collection through a reward mechanism. The collected user data is preprocessed and used as supplementary data for model training, enabling iterative optimization of congestion prediction models and governance strategies.

[0083] Here, the preprocessed user data serves as supplementary data for model training. Its purpose is to fill the data gaps in the system's data collection, enrich the scenario samples for model training, and improve the model's adaptability to real-world traffic conditions. This perfectly solves the problems of limited model samples and insufficient capture of local / sudden / special scenarios in existing technologies. For example, by filling the traffic data gaps in sensor deployment blind spots, it enhances the full road network coverage of congestion prediction models; by supplementing real-time data on sudden traffic events, it improves the model's response to emergencies; and by supplementing user behavior feature data, it optimizes the accuracy of personalized route recommendations and governance strategies.

[0084] An embodiment of the present invention also provides a smart traffic congestion management system based on multimodal data fusion, used to implement the aforementioned smart traffic congestion management method based on multimodal data fusion, which includes a data acquisition and fusion module, a congestion prediction module, and a management decision module.

[0085] The data acquisition and fusion module is used to acquire basic traffic data and related traffic data from multiple dimensions, preprocess the basic traffic data and related traffic data, perform multimodal fusion on the preprocessed basic traffic data and related traffic data to obtain traffic state fusion data, as well as road network spatial data and historical congestion time series data, and construct a traffic state spatiotemporal feature map based on the road network spatial data and historical congestion time series data.

[0086] The congestion prediction module is used to dynamically capture key congestion influencing factors in different road sections and time periods through the attention mechanism based on traffic condition fusion data, and obtain congestion prediction results; the governance decision module is used to obtain multiple sets of candidate traffic governance strategies based on the congestion prediction results, and obtain the optimal governance strategy based on the multiple sets of candidate traffic governance strategies.

[0087] As a preferred technical solution, the intelligent traffic congestion management system also includes a cross-departmental collaborative management module and a user feedback and interaction module.

[0088] The cross-departmental collaborative governance module is used to publish the optimal governance strategy on the blockchain, so as to achieve full-process traceability of shared data among multiple departments during the execution of the governance strategy, and to enable multi-departmental collaboration to execute the optimal governance strategy.

[0089] The user feedback interaction module receives real-time congestion information reported by users. Based on the real-time congestion information, basic traffic data, and related traffic data, it provides users with personalized route recommendations and incentivizes users to participate in data collection through a reward mechanism. The collected user data is preprocessed and used as supplementary data for model training, enabling iterative optimization of the congestion prediction model and governance strategies.

[0090] The governance decision module is also used to adapt the congestion prediction model to different new traffic scenarios based on transfer learning, and to iteratively update the parameters of the congestion prediction model at preset time intervals. At the same time, by constructing a digital twin virtual traffic environment, multiple sets of candidate traffic governance strategies are input into the digital twin virtual traffic environment for effect simulation, and the optimal governance strategy is selected based on the simulation results.

[0091] In summary, the intelligent traffic congestion management system described in this invention comprehensively addresses the technical shortcomings of existing technologies in data fusion, model prediction, strategy governance, and collaborative management by constructing an integrated intelligent traffic congestion management system that combines multimodal data fusion and collection, adaptive dynamic prediction, reinforcement learning decision-making, blockchain collaboration, and user feedback. It can achieve full-process, closed-loop dynamic management of urban traffic congestion and improve the accuracy of congestion prediction.

[0092] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0093] The embodiments described above are merely illustrative of several implementations of the present invention, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the invention patent. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these all fall within the protection scope of the present invention. Therefore, the protection scope of this invention patent should be determined by the appended claims.

Claims

1. A smart traffic congestion management method based on multimodal data fusion, characterized in that, The intelligent traffic congestion management method includes the following steps: Acquire basic traffic data and related traffic data from multiple different dimensions, and preprocess the basic traffic data and related traffic data; A data fusion unit is constructed, and based on the data fusion unit, multimodal fusion is performed on the preprocessed basic traffic data and related traffic data to obtain traffic status fusion data; Obtain road network spatial data and historical congestion time series data, and construct a spatiotemporal feature map of traffic status based on the road network spatial data and historical congestion time series data; A congestion prediction model is constructed, using traffic state fusion data as input. The model dynamically captures key congestion influencing factors in different road sections and time periods through an attention mechanism to obtain congestion prediction results. Multiple candidate traffic management strategies are obtained based on congestion prediction results, and the optimal management strategy is obtained based on these multiple candidate strategies.

2. The intelligent traffic congestion management method based on multimodal data fusion as described in claim 1, characterized in that, The intelligent traffic congestion management method also includes the following steps: The optimal governance strategy is published on the blockchain to achieve full-process traceability of shared data among multiple departments during the implementation of the governance strategy; Multi-departmental collaboration is essential to implement optimal governance strategies.

3. The intelligent traffic congestion management method based on multimodal data fusion as described in claim 2, characterized in that, The intelligent traffic congestion management method also includes the following steps: It receives real-time traffic congestion information reported by users and provides personalized route recommendations based on real-time traffic congestion information, basic traffic data, and related traffic data. By incentivizing users to participate in data collection through a reward mechanism, the collected user data is preprocessed and used as supplementary data for model training, thereby enabling iterative optimization of congestion prediction models and governance strategies.

4. The intelligent traffic congestion management method based on multimodal data fusion as described in claim 3, characterized in that, The intelligent traffic congestion management method also includes the following steps: For different new traffic scenarios, the congestion prediction model is adapted based on transfer learning; The parameters of the congestion prediction model are iteratively updated at preset time intervals.

5. The intelligent traffic congestion management method based on multimodal data fusion as described in claim 4, characterized in that, The specific method for obtaining the optimal traffic management strategy from multiple candidate traffic management strategies includes the following steps: Construct a digital twin virtual traffic environment and input multiple sets of candidate traffic management strategies into the digital twin virtual traffic environment to conduct effect simulation; The optimal governance strategy is selected based on the results of the preliminary simulation.

6. A smart traffic congestion management system based on multimodal data fusion, used to implement the smart traffic congestion management method based on multimodal data fusion as described in any one of claims 1-5, characterized in that, The intelligent traffic congestion management system includes: The data acquisition and fusion module is used to acquire basic traffic data and related traffic data from multiple different dimensions, preprocess the basic traffic data and related traffic data, perform multimodal fusion on the preprocessed basic traffic data and related traffic data, acquire traffic state fusion data, acquire road network spatial data and historical congestion time series data, and construct a traffic state spatiotemporal feature map based on the road network spatial data and historical congestion time series data. The congestion prediction module is used to dynamically capture key congestion influencing factors in different road sections and time periods through attention mechanism based on traffic condition fusion data to obtain congestion prediction results. The governance decision module is used to obtain multiple sets of candidate traffic governance strategies based on congestion prediction results, and to obtain the optimal governance strategy based on the multiple sets of candidate traffic governance strategies.

7. The intelligent traffic congestion management system based on multimodal data fusion as described in claim 6, characterized in that, The intelligent traffic congestion management system also includes: The cross-departmental collaborative governance module is used to publish the optimal governance strategy on the blockchain, so as to achieve full-process traceability of shared data among multiple departments during the execution of the governance strategy, and to enable multi-departmental collaboration to execute the optimal governance strategy.

8. The intelligent traffic congestion management system based on multimodal data fusion as described in claim 7, characterized in that, The intelligent traffic congestion management system also includes: The user feedback interaction module is used to receive real-time congestion information reported by users. Based on the real-time congestion information, basic traffic data, and related traffic data, it provides users with personalized route recommendations and incentivizes users to participate in data collection through a reward mechanism. The collected user data is preprocessed and used as supplementary data for model training to achieve iterative optimization of the congestion prediction model and governance strategies.

9. A smart traffic congestion management system based on multimodal data fusion as described in claim 8, characterized in that, The governance decision-making module is also used for: For different new traffic scenarios, the congestion prediction model is adapted based on transfer learning, and the parameters of the congestion prediction model are iteratively updated at preset time intervals.

10. A smart traffic congestion management system based on multimodal data fusion as described in claim 9, characterized in that, The governance decision-making module is also used for: A digital twin virtual traffic environment is constructed, and multiple sets of candidate traffic management strategies are input into the digital twin virtual traffic environment to conduct effect simulations. The optimal management strategy is then selected based on the simulation results.

Images