Intelligent traffic monitoring system and method based on multi-source data fusion

Abstract

The invention relates to the technical field of smart traffic, and discloses a smart traffic monitoring system and method based on multi-source data fusion, and the system comprises a multi-source data collection module, a spatial-temporal feature fusion engine, a hierarchical decision core, a decision execution module, and an online learning module. The method corresponds to the system. According to the method, heterogeneous traffic data are unified under the same time-space reference through a multi-modal fusion technology, and comprehensive and accurate road network state feature representation is constructed; the hierarchical decision-making core forms a closed-loop decision-making link from event identification to control strategy generation through close cooperation of an event sensing layer and a regional optimization layer, and realizes rapid response and accurate control of traffic abnormity; on-line learning continuously optimizes decision logic based on complete historical operation data so as to adapt to a continuously changing traffic environment; finally, the accuracy of traffic state perception and the timeliness of decision response are improved, and reliable technical guarantee is provided for efficient management and control of intelligent traffic.

Description

Technical Field

[0001] This invention relates to the field of intelligent transportation technology, specifically to an intelligent transportation monitoring system and method based on multi-source data fusion. Background Technology

[0002] In the development of intelligent transportation systems, achieving accurate traffic condition perception and intelligent decision-making has always been a core technical challenge. Existing solutions have significant limitations in data processing and decision-making mechanisms: on the one hand, multi-source traffic data (including floating car trajectories, fixed detector data, video images, etc.) are difficult to integrate effectively due to their heterogeneity and inconsistent spatiotemporal scales, leading to biases and delays in traffic situation perception; on the other hand, the perception layer and decision-making layer are often independent, with event recognition and control strategy generation disconnected, making it difficult for the system to form a closed-loop optimization from perception to decision. Specifically, due to the lack of a unified feature representation method, traffic information from different sources cannot be effectively correlated under a unified spatiotemporal benchmark; simultaneously, the decision-making process is fragmented, making it difficult to dynamically generate collaborative control strategies based on real-time traffic events. This separation between perception and decision-making causes existing systems to respond slowly to complex and ever-changing traffic conditions, failing to achieve adaptive optimization and collaborative control of regional traffic flow, thus hindering the overall performance improvement of intelligent transportation systems. Summary of the Invention

[0003] The purpose of this invention is to provide a smart traffic monitoring system and method based on multi-source data fusion to solve the technical problems mentioned in the background.

[0004] To achieve the above objectives, the present invention discloses the following technical solutions: In a first aspect, the present invention discloses an intelligent traffic monitoring system based on multi-source data fusion, the system comprising: The multi-source data acquisition module is used to collect raw data of various traffic elements in the traffic network; A spatiotemporal feature fusion engine is used to perform fusion processing on the original data. The fusion processing includes map matching of floating car data, spatiotemporal alignment of roadside sensor data, and convolutional neural network feature extraction of video stream data to obtain processed multi-source data. The multi-source data is then fused in a multimodal manner to generate a road network state feature tensor under a unified spatiotemporal reference. The hierarchical decision-making core includes an event perception layer and a regional optimization layer; the event perception layer is used to analyze the road network state feature tensor to identify abnormal traffic events; the regional optimization layer is used to generate a collaborative control strategy for regional traffic flow based on the identification results of the abnormal traffic events. The decision execution module is used to execute the collaborative control strategy and output device control commands to the traffic control equipment; The online learning module is used to optimize the decision logic of the hierarchical decision core through online learning based on historical operational data. The historical operational data includes historical road network state feature tensors, historical traffic anomaly event identification results, historical collaborative control strategies, and traffic state change data after the implementation of collaborative control strategies.

[0005] Optionally, the identification of the traffic anomaly event includes: The event perception layer inputs the road network state feature tensor and its corresponding road network topology into a preset spatiotemporal graph convolutional network, and the spatiotemporal graph convolutional network outputs the identification result of traffic anomaly events after performing operations on the input data.

[0006] Optionally, performing operations on the input data includes: Perform a graph convolution operation on the input road network state feature tensor and the road network topology to obtain a node feature matrix representing the spatial dependence of the road network; The node feature matrix is ​​input into a gated recurrent unit network to obtain a node state sequence with time dynamics. The node state sequence is input into a fully connected classification layer to obtain the classification results of traffic anomaly events for each node's corresponding road segment.

[0007] Optionally, the spatiotemporal graph convolutional network is provided with a dynamic graph structure update mechanism, which includes: Based on real-time traffic flow data, calculate the weights of the connecting edges between adjacent nodes in the road network topology; The road network topology is dynamically updated based on the calculated weights. The graph convolution operation is performed using the updated road network topology.

[0008] Optionally, the generation of the collaborative control strategy includes: The regional optimization layer uses the identification results of the traffic anomaly events as the global state, and fuses them with the local observation states of each agent in the preset signal control model. Through policy network calculation, it outputs a coordinated signal timing scheme; and / or, The regional optimization layer will generate the display content and release strategy of the traffic guidance screen based on the identification results of the traffic anomaly event and the real-time traffic flow data, through logical reasoning using a pre-set decision tree model. The coordinated signal timing scheme and / or the display content and publishing strategy of the traffic guidance screen shall be used as the coordinated control strategy.

[0009] Optionally, the policy network computation includes: Each agent concatenates its local observation state with the global state to obtain the agent's comprehensive state vector; The integrated state vector is input into the Actor policy network of the agent to obtain the probability distribution of signal phase actions; Based on the probability distribution, the switching action of the signal phase or the adjustment action of the green light duration are selected and output to form the coordinated signal timing scheme.

[0010] Optionally, the logical reasoning includes: The type, severity level, and real-time traffic flow of the upstream road segment of the traffic anomaly are used as input features and fed into the decision tree model. The decision tree model matches predefined rule branches based on the input features and outputs the corresponding guidance screen text code and the target guidance screen device identifier. Based on the text encoding of the guidance screen and the device identifier of the target guidance screen, a traffic guidance screen control command with display content and publishing range is generated to constitute the display content and publishing strategy of the traffic guidance screen.

[0011] Optionally, the output of the control command includes: When the cooperative control strategy includes a cooperative signal timing scheme, the decision execution module inputs the cooperative control strategy into a traffic flow simulator based on a cellular transmission model to obtain the simulated total regional delay. The decision execution module compares the simulated total regional delay with the current total regional delay, and only when the simulated total regional delay is less than the current total regional delay is the cooperative signal timing scheme issued as a device control command to the traffic control device. When the collaborative control strategy includes the display content and publishing strategy of the traffic guidance screen, the display content and publishing strategy of the traffic guidance screen are sent to the traffic control device as a device control command.

[0012] Optionally, the generation of the road network state feature tensor includes: The spatiotemporal feature fusion engine maps the floating car data after map matching, the roadside sensor data after spatiotemporal alignment, and the video stream data extracted by the convolutional neural network to the same high-dimensional feature space to obtain high-dimensional features. Based on the real-time coverage and historical accuracy of the floating car data, the roadside sensor data, and the video stream data, calculate the intermodal attention weights; The high-dimensional features after mapping are weighted and nonlinearly transformed by the intermodal attention weights to generate the road network state feature tensor.

[0013] Secondly, this application discloses a method for use in the intelligent traffic monitoring system based on multi-source data fusion as described above, the method comprising: The data acquisition and fusion stage includes: acquiring raw data of various traffic elements in the traffic network; performing fusion processing on the raw data, including map matching of floating car data, spatiotemporal alignment of roadside sensor data, and convolutional neural network feature extraction of video stream data; and performing multimodal fusion on the processed multi-source data to generate a road network state feature tensor under a unified spatiotemporal reference. The hierarchical decision-making and strategy generation stage includes: analyzing the road network state feature tensor through the event perception layer to identify abnormal traffic events; and generating a collaborative control strategy for regional traffic flow based on the identified abnormal traffic events through the regional optimization layer. The strategy execution and feedback optimization phase includes: executing the collaborative control strategy and outputting corresponding equipment control commands to the traffic control equipment; collecting historical operating data, including historical road network state feature tensors, historical traffic anomaly event identification results, historical collaborative control strategies, and traffic state change data after execution; and optimizing the decision logic of the hierarchical decision-making based on the historical operating data through online learning.

[0014] Beneficial Effects: The intelligent traffic monitoring system and method based on multi-source data fusion of this invention effectively overcomes the core problems of insufficient multi-source data fusion and the disconnect between perception and decision-making in existing technologies by constructing a collaborative architecture of a spatiotemporal feature fusion engine and a hierarchical decision core. Specifically, this application unifies heterogeneous traffic data under the same spatiotemporal benchmark through multimodal fusion technology, constructing a comprehensive and accurate road network state feature representation, fundamentally solving the perception bias problem caused by dispersed data sources and inconsistent formats; the hierarchical decision core, through the close cooperation between the event perception layer and the regional optimization layer, forms a closed-loop decision-making link from event identification to control strategy generation, realizing rapid response and precise control of traffic anomalies; online learning continuously optimizes the decision logic based on complete historical operating data, enabling the system to have self-evolution capabilities and adapt to the ever-changing traffic environment. Under the combined effect of this collaborative architecture, the accuracy of traffic state perception, the timeliness of decision response, and the overall adaptability of the system are improved, providing a reliable technical guarantee for achieving efficient intelligent traffic management. Attached Figure Description

[0015] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0016] Figure 1 A structural block diagram of a smart traffic monitoring system based on multi-source data fusion provided in an embodiment of this application; Figure 2 A flowchart illustrating the intelligent traffic monitoring method based on multi-source data fusion provided in this application embodiment. Detailed Implementation

[0017] To facilitate understanding of the technical solutions provided in the embodiments of this application, the background technology involved in the embodiments of this application will be described below.

[0018] In the field of intelligent transportation, with the acceleration of urbanization and the continuous expansion of transportation networks, the dynamic changes of traffic elements (vehicles, pedestrians, road facilities, etc.) are becoming increasingly complex. How to achieve accurate perception of traffic conditions and intelligent decision-making based on perception results has become the core demand for improving traffic control efficiency, alleviating traffic congestion, and ensuring traffic safety. It is also a key technical challenge that urgently needs to be overcome in the development of intelligent transportation systems.

[0019] Current intelligent traffic monitoring technologies have significant limitations in data processing capabilities and decision-making mechanism design, making them difficult to adapt to the actual needs of complex traffic scenarios, as detailed below: On the one hand, the data sources in the transportation network exhibit significant diversification, encompassing trajectory data generated by floating cars (such as discrete mobile data like real-time vehicle location, speed, and direction), continuous data from fixed points such as traffic flow and lane occupancy collected by roadside fixed sensors (such as geomagnetic detectors, coil detectors, and microwave detectors), and visual data from video streams collected by surveillance cameras (such as image information like vehicle appearance, pedestrian behavior, and intersection scenes). However, these multi-source traffic data exhibit inherent heterogeneity and spatiotemporal inconsistencies: in terms of data type, floating car data focuses on individual vehicle dynamics, sensor data focuses on the macroscopic state of road segments, and video stream data focuses on visual scene details, with significant differences in format and semantic dimensions; in terms of spatiotemporal attributes, the sampling interval of floating car data is not fixed (affected by vehicle driving status), the timestamps of different sensors may have synchronization deviations, and the spatial reference benchmarks (such as coordinate systems and coverage areas) of each data source are inconsistent. Existing technologies lack effective deep integration methods, making it impossible to perform targeted preprocessing on these heterogeneous data (such as accurate matching of floating car trajectories with maps, spatiotemporal alignment of sensor data, and effective feature extraction from video streams), and also making it difficult to establish a unified spatiotemporal benchmark to link and integrate scattered data. This leads to biases in the perception of traffic conditions (such as missing local congestion areas and misidentifying abnormal events) and lags (such as the inability to capture sudden changes in traffic flow in a timely manner).

[0020] On the other hand, in existing technologies, the perception layer and the decision-making layer are often independent of each other, failing to form an efficient collaborative closed-loop mechanism. Specifically, the perception layer only completes the collection and simple processing of single-dimensional data, lacking a unified feature representation method, and cannot transform multi-source data into a unified feature form that can comprehensively reflect the road network status. Meanwhile, the decision-making layer often adopts fixed and fragmented decision-making logic, with event recognition (such as traffic accidents, traffic violations, road construction, and other abnormal events) and control strategy generation completely disconnected. It is unable to accurately identify abnormal events based on the integrated road network status, nor can it dynamically generate collaborative control strategies (such as signal timing optimization, lane function adjustment, and guidance information dissemination) adapted to regional traffic flow based on real-time recognition results. This design of separation between perception and decision-making leads to slow decision response and unscientific control strategies when existing systems face complex and ever-changing traffic conditions. It is unable to achieve adaptive optimization and collaborative control of regional traffic flow (e.g., after an accident at an intersection, it is unable to quickly coordinate with surrounding intersections to adjust signal timing, leading to the spread of congestion), ultimately restricting the overall operational efficiency of the intelligent transportation system.

[0021] Therefore, existing technologies suffer from insufficient multi-source data fusion capabilities and a disconnect between perception and decision-making mechanisms, resulting in inaccurate traffic status perception, delayed decision response, and limited control effectiveness. To address this, this embodiment provides a technical solution that overcomes these limitations, achieving deep fusion of multi-source data and intelligent operation throughout the entire perception-decision-optimization process.

[0022] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present application. Secondly, in this document, the term "comprising" is intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements, but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes the element.

[0023] This embodiment provides, in its first aspect, a smart traffic monitoring system based on multi-source data fusion, such as... Figure 1 As shown, the system includes a multi-source data acquisition module, a spatiotemporal feature fusion engine, a hierarchical decision core, a decision execution module, and an online learning module, as detailed below.

[0024] The multi-source data acquisition module is used to collect raw data of various traffic elements in the traffic network.

[0025] In practical implementation, various traffic elements are considered, including vehicles (driving status, location, etc.), roads (road segment capacity, number of lanes, etc.), traffic participants (pedestrians, non-motorized vehicles, etc.), and traffic equipment status (signal light operating parameters, detector online status, etc.). The collected raw data specifically includes: floating car data (collected via GPS / BeiDou positioning terminals installed in private vehicles, taxis, ride-hailing vehicles, etc., including timestamps, latitude and longitude, instantaneous speed, and driving direction), roadside sensor data (collected via roadside geomagnetic detectors for parking events, loop detectors for traffic flow / occupancy, microwave radar for long-distance vehicle speeds, and lidar for three-dimensional traffic flow distribution), video stream data (collected via roadside high-definition cameras for real-time image sequences of intersections / road segments), as well as real-time timing data from traffic signals and violation records from electronic police systems.

[0026] The spatiotemporal feature fusion engine is used to fuse raw data. The fusion process includes map matching of floating car data, spatiotemporal alignment of roadside sensor data, and convolutional neural network feature extraction of video stream data to obtain processed multi-source data. The multi-source data is then fused in a multimodal manner to generate a road network state feature tensor under a unified spatiotemporal reference.

[0027] In practical implementation, map matching of floating car data is achieved using a Hidden Markov Model (HMM): road segments on the electronic road network map are used as hidden states, and floating car positioning points are used as observations. By calculating the spatial distance probability between the positioning point and the road segment (observation probability) and the probability of reasonable turning between road segments (transition probability), the matching of positioning points to optimal road segments is achieved. Spatiotemporal alignment of roadside sensor data includes time alignment (calibrating the timestamps of each sensor based on the NTP network time protocol) and spatial alignment (mapping all sensor data to a unified Gaussian plane coordinate system and correcting the installation position deviations of different sensors through coordinate transformation formulas). Convolutional neural network feature extraction of video stream data is implemented using the CSP-Darknet53 network, extracting visual features such as vehicle density, queue length, and pedestrian aggregation through a multi-scale feature fusion layer.

[0028] The hierarchical decision-making core consists of an event perception layer and a regional optimization layer. The event perception layer is used to analyze the road network state feature tensor to identify abnormal traffic events. The regional optimization layer is used to generate a collaborative control strategy for regional traffic flow based on the identification results of abnormal traffic events.

[0029] In practice, traffic anomalies can include events such as traffic accidents, illegal parking, road construction, and traffic congestion. Coordinated control strategies can include signal timing adjustments, temporary lane function switching, and the dissemination of traffic guidance information.

[0030] The decision execution module is used to execute the collaborative control strategy and output equipment control commands to the traffic control equipment.

[0031] In practice, traffic control equipment includes traffic signals, variable lane indicators, traffic guidance screens, ramp controllers, etc.

[0032] The online learning module is used to optimize the decision-making logic of the hierarchical decision-making core by learning from historical operational data. The historical operational data includes historical road network state feature tensors, historical traffic anomaly event identification results, historical collaborative control strategies, and traffic state change data after the implementation of collaborative control strategies.

[0033] In practice, online learning employs a policy gradient algorithm based on deep reinforcement learning, with the maximization of regional traffic efficiency as the reward function (the reward value is positively correlated with the regional average vehicle speed and negatively correlated with the total delay time), and the parameters of the decision model are continuously updated through iterative updates.

[0034] Based on the above, the intelligent traffic monitoring system based on multi-source data fusion in this embodiment integrates heterogeneous and multi-scale raw data from the traffic network through a multi-source data acquisition module. This overcomes the limitations of the one-sidedness of information from a single data source and lays a data foundation for comprehensive perception of traffic conditions. Specifically, the spatiotemporal feature fusion engine transforms scattered data into a road network state feature tensor under a unified spatiotemporal benchmark through targeted preprocessing (map matching to solve the floating car positioning drift problem, spatiotemporal alignment to eliminate spatiotemporal bias in sensor data, and convolutional neural networks to extract deep features from videos) and multimodal fusion. This achieves deep integration of traffic information and avoids cognitive bias caused by data fragmentation. The hierarchical decision-making core closely links abnormal event identification with control strategy generation through a hierarchical mechanism of event perception and regional optimization, solving the problem of disconnect between perception and decision-making in traditional systems. The online learning module continuously optimizes the decision logic based on historical data, enabling the system to adapt to dynamic changes in traffic conditions (such as morning and evening rush hour patterns and the impact of sudden weather). Thus, the synergistic effect of each module realizes a closed loop from data collection and integrated perception to intelligent decision-making and execution optimization, effectively improving the accuracy and timeliness of traffic anomaly identification, ensuring the scientific nature of control strategies, and ultimately promoting the coordinated optimization of regional traffic flow and the improvement of overall traffic efficiency.

[0035] As an optional implementation of this embodiment, the identification of traffic anomalies includes: The event perception layer inputs the road network state feature tensor and its corresponding road network topology into a preset spatiotemporal graph convolutional network, and the spatiotemporal graph convolutional network outputs the recognition results of traffic anomaly events after performing operations on the input data.

[0036] In practical implementation, the road network topology is a graph structure. Representation: Node set Each node in the set corresponds to a basic road segment (such as a continuous road segment within 500 meters in length) or an intersection in the road network. Node attributes include static information such as road segment length, number of lanes, and design speed. It represents the connection relationships between nodes (such as the connection between the beginning and end of adjacent road segments, and the connection between intersections and related road segments). Edge attributes include turning restrictions (such as prohibiting left turns) and connection distances. Spatiotemporal graph convolutional networks can be traditional graph convolutional networks; or they can be deep learning models that introduce time dimension modeling capabilities on the basis of traditional graph convolutional networks. Their input is tensor data with spatial dimension (node ​​features) - temporal dimension (temporal changes), and the output is the abnormal event type corresponding to each node (such as "no abnormality", "minor congestion", "serious accident", etc., a total of 8 predefined event types).

[0037] Based on the above, the intelligent traffic monitoring system based on multi-source data fusion in this embodiment combines the road network state feature tensor with the road network topology and inputs it into a spatiotemporal graph convolutional network. This allows the system to simultaneously capture the spatial correlation (e.g., the propagation relationship of congestion between adjacent road segments) and temporal dynamics (e.g., the temporal changes from the occurrence to the spread of congestion). Compared to traditional identification methods based solely on a single time series or spatial distribution, this approach fully utilizes the topological connectivity characteristics of the road network and the spatiotemporal evolution of traffic flow. This enables abnormal event identification to not only rely on the state characteristics of local road segments but also to make comprehensive judgments based on the state changes of surrounding related road segments. This effectively reduces the misjudgment rate caused by local data noise (e.g., temporary sensor malfunctions) and improves the accuracy and timeliness of identifying complex traffic anomalies (e.g., regional congestion, chain accidents).

[0038] Based on the aforementioned identification of traffic anomalies, as a further optional implementation of this embodiment, operations are performed on the input data, including: Perform graph convolution operation on the input road network state feature tensor and road network topology to obtain a node feature matrix representing the spatial dependence of the road network; The node feature matrix is ​​input into a gated recurrent unit (GRU) network to obtain a node state sequence with time dynamics. The node state sequence is input into a fully connected classification layer to obtain the classification results of traffic anomaly events for each node's corresponding road segment.

[0039] In practice, the graph convolution operation is based on an improved Chebyshev polynomial approximation algorithm, and the calculation formula is as follows: in, The output node feature matrix, The order is the polynomial (values ​​3-5, balancing computational load and feature extraction capability). For learnable convolution kernel parameters, For Chebyshev polynomials ( , , ), The normalized Laplace matrix ( , For the graph Laplace matrix, for The largest eigenvalue, (the identity matrix) The input node feature matrix (derived from a spatial dimension slice of the road network state feature tensor).

[0040] Secondly, the gate control loop unit controls the flow of information by updating and resetting the gates, and its state update formula is as follows: in, To update the door, To reset the door, It is the sigmoid activation function. Element-wise multiplication Slice the node feature matrix at time t. The hidden state at time t-1 In the candidate hidden state, The output state at time t (i.e., an element of the node state sequence). , , These are the input weight matrix of the update gate, the hidden state weight matrix, and the bias vector, respectively. , , These are the input weight matrix of the reset gate, the hidden state weight matrix, and the bias vector, respectively. , , Let be the input weight matrix, the hidden state weight matrix, and the bias vector, respectively, and each... All of these are learnable parameters of the GRU.

[0041] In addition, the fully connected classification layer contains two hidden layers (with 128 and 64 neurons respectively, using the ReLU activation function) and one output layer (with the number of neurons equal to the number of abnormal event categories, using the Softmax activation function to output the probability of each category).

[0042] Based on the above, the intelligent traffic monitoring system based on multi-source data fusion in this embodiment effectively mines the spatial dependencies between nodes in the road network topology (such as the impact of main road congestion on related branch roads) through graph convolution operations using an improved Chebyshev multinomial algorithm. The generated node feature matrix can accurately characterize the spatial correlation characteristics of different road segments. The gated cyclic unit network selectively retains historical state information through a gating mechanism, capturing the dynamic evolution of traffic conditions over time (such as the formation and dissipation of morning rush hour congestion), expanding spatial features into a spatiotemporally fused node state sequence. Finally, a fully connected classification layer maps high-dimensional features to specific abnormal event categories, achieving end-to-end identification. This process, through the progressive processing of spatial feature extraction, temporal dynamic modeling, and category mapping, fully integrates the spatiotemporal characteristics of traffic conditions, improving the accuracy and robustness of identifying abnormal traffic events (especially those with spatiotemporal diffusion characteristics) compared to single-dimensional modeling methods.

[0043] Based on the aforementioned graph convolution operations, as a further optional implementation of this embodiment, the spatiotemporal graph convolutional network is provided with a dynamic graph structure update mechanism, which includes: Calculate the weights of the connecting edges between adjacent nodes in the road network topology based on real-time traffic flow data. The road network topology is dynamically updated based on the calculated weights. Perform graph convolution operations using the updated road network topology.

[0044] In practical implementation, the formula for calculating the weight of the connecting edge is: in, The calculated edge weights (range 0-1) between node i and node j. , and Weighting coefficients ( Determined through offline training, such as , , ); The traffic flow from node i to node j per unit time (which can be counted by roadside sensors). This represents the highest traffic volume in history. , Let be the average vehicle speeds of nodes i and j, respectively. Design the maximum vehicle speed for the road network; The historical average edge weights (reflecting long-term correlation patterns).

[0045] Secondly, the update of the road network topology can specifically be: when ( When a preset threshold (e.g., 0.2, determined by the ROC curve) is reached, the edge is temporarily removed (indicating a weak current correlation between the two nodes); when... When the edge is removed, its weight is updated.

[0046] Based on the above, the intelligent traffic monitoring system based on multi-source data fusion in this embodiment employs a dynamic graph structure update mechanism that calculates edge weights using real-time traffic flow data. This allows the road network topology to dynamically adjust according to traffic conditions. For example, during peak hours, the frequent exchange of traffic flow between main roads and secondary roads increases edge weights, strengthening the connection between them in the topology. Conversely, during off-peak hours or when an accident occurs on a certain road segment, the traffic flow on the associated road segment decreases sharply, reducing edge weights and weakening irrelevant connections in the topology. This dynamic adjustment enables graph convolution operations to focus on currently strongly correlated node pairs, avoiding interference from invalid connections in the static topology on feature extraction (such as the weak connection between main roads and suburban secondary roads during off-peak hours). This enhances the sensitivity of the spatiotemporal graph convolutional network to real-time traffic anomalies (such as quickly identifying sudden changes in traffic flow on associated road segments caused by accidents) and strengthens the model's adaptability to complex traffic scenarios.

[0047] As an optional implementation method of this embodiment, the generation of the cooperative control strategy includes: The regional optimization layer uses the identification results of traffic anomalies as the global state, integrates it with the local observation states of each agent in the pre-defined signal control model, and outputs a coordinated signal timing scheme through policy network calculation; and / or, The regional optimization layer will use the identification results of traffic anomalies and real-time traffic flow data to perform logical reasoning through a pre-set decision tree model to generate the display content and release strategy of the traffic guidance screen. The coordinated signal timing scheme and / or the display content and publishing strategy of traffic guidance screens shall be used as the coordinated control strategy.

[0048] In practice, the signal control model is based on multi-agent reinforcement learning. Each agent corresponds to a traffic signal controller at an intersection, and the local observation status includes the queue length, average waiting time, and remaining green light time for each phase at the intersection.

[0049] Secondly, real-time traffic flow data includes traffic volume, average speed, and density of vehicles on roads upstream and downstream of the incident site. The root node of the decision tree model represents the type of abnormal event (e.g., accident / congestion / construction), the intermediate nodes include severity level (levels 1-3), and impact range (within 500 meters / 500-1000 meters / over 1000 meters), and the leaf nodes are guidance strategy templates (e.g., "An accident is 500 meters ahead; it is recommended to detour via XX road").

[0050] Based on the above, the intelligent traffic monitoring system based on multi-source data fusion in this embodiment integrates global abnormal events and local states through a multi-agent signal control model. The resulting collaborative signal timing scheme can break the limitations of individual intersections operating independently (e.g., avoiding congestion at related intersections due to excessively long green lights at a certain intersection), and achieve coordinated optimization of intersection signals within the region (e.g., extending red lights at intersections upstream of an accident site and extending green lights at intersections downstream of an accident site to accelerate traffic flow). Furthermore, the traffic guidance strategy generation based on decision trees can quickly match the optimal guidance scheme according to the specific characteristics (type, level, and scope of impact) of abnormal events, guiding vehicles to detour in advance through guidance screens to alleviate pressure at the incident site. The combination of these two strategies (or using one at a time) forms a collaborative management mechanism of signal control and traffic guidance. Compared to traditional isolated management methods, this can more efficiently manage regional traffic flow and reduce the impact of abnormal events on overall traffic efficiency.

[0051] Based on the aforementioned collaborative control strategy generation, as a further optional implementation method in this embodiment, policy network computation includes: Each agent concatenates its local observation state with its global state to obtain the agent's overall state vector; The integrated state vector is input into the Actor policy network of the agent to obtain the probability distribution of signal phase actions; Based on probability distribution, select and output the switching action of signal phase or the adjustment action of green light duration to form a coordinated signal timing scheme.

[0052] In practice, the local observation state is an 8-dimensional vector (containing queue lengths in 4 phases, average vehicle speeds in 2 directions, current phase, and remaining green light duration), and the global state is a 6-dimensional vector (containing abnormal event type code, severity level, incident location coordinates, radius of influence, average delay in the current area, and duration of the abnormality). After splicing, the overall state vector has 14 dimensions.

[0053] Secondly, the Actor policy network is a 3-layer fully connected neural network (with 64 and 32 neurons in the hidden layers, respectively, and using the LeakyReLU activation function). The output layer uses the Gumbel-Softmax function to generate the probability distribution of discrete actions (the action space includes 4 actions such as maintaining the current phase, switching to the next phase, extending the current green light by 5 seconds, and shortening the current green light by 3 seconds).

[0054] Furthermore, an ε-greedy strategy is specifically adopted to select actions: based on probability. ( The probability of selecting the action with the highest probability is linearly decreased from 0.3 to 0.05 during training iterations. Randomly select actions to balance exploration and exploitation.

[0055] Based on the above, the intelligent traffic monitoring system based on multi-source data fusion in this embodiment integrates the local details and global anomaly information of the agents into the comprehensive state vector. This allows each intersection's decision-making to consider both its own traffic conditions (such as queue length) and respond to global anomalies (such as the potential impact of distant accidents on the intersection), avoiding one-sided decision-making. The Actor policy network uses a deep neural network to perform nonlinear mapping of high-dimensional states, generating an action probability distribution that adapts to the dynamic changes in complex traffic scenarios (such as differences in optimal actions under different time periods and anomaly types). The ε-greedy policy, while ensuring the utilization of current optimal experience, retains the possibility of exploring new actions, enabling the policy network to continuously learn better cooperative timing schemes. Ultimately, through distributed decision-making and implicit cooperation among the agents, dynamic optimization of regional signal timing is achieved, effectively improving traffic flow dispersal efficiency under abnormal events.

[0056] Similarly, based on the aforementioned collaborative control strategy generation, as a further optional implementation method of this embodiment, logical reasoning includes: The type, severity level, and real-time traffic flow of the traffic anomaly upstream of the incident point are used as input features and fed into the decision tree model. The decision tree model matches predefined rule branches based on input features and outputs the corresponding text code of the guidance screen and the device identifier of the target guidance screen; Based on the text encoding of the traffic guidance screen and the device identifier of the target traffic guidance screen, control instructions with display content and publishing range are generated to form the display content and publishing strategy of the traffic guidance screen.

[0057] In practice, abnormal events are classified into 5 types (codes 1-5: accident / congestion / construction / temporary control / other), and the severity level is divided into 3 levels (level 1: affecting 1 road segment; level 2: affecting 2-3 road segments; level 3: affecting 4 or more road segments). Real-time traffic flow is a normalized value (0-1, where 0 represents no vehicles and 1 represents saturated flow).

[0058] Secondly, the rule branch example is: if "type = accident (1)" and "level = level 2" and "upstream flow > 0.7", then the matching branch is "guidance text code = 102 (text content: 'An accident occurred 800 meters ahead, traffic is slow, it is recommended to exit from XX exit in advance and detour via XX road')", "target guidance screen identifier = S301, S302 (guidance screens 1 km and 2 km upstream of the incident point)".

[0059] In addition, the traffic guidance screen control instructions include parameters such as text content, display duration (set according to the level of abnormality: 3 minutes for level 1, 5 minutes for level 2, and 10 minutes for level 3), and refresh rate (1 time / 30 seconds).

[0060] Based on the above, the intelligent traffic monitoring system based on multi-source data fusion in this embodiment focuses the input features of the decision tree model on the core attributes (type, level) of abnormal events and the upstream traffic with the highest impact correlation, ensuring the relevance and effectiveness of the reasoning. Predefined rule branches, designed based on traffic engineering experience and historical cases, can quickly match guidance strategies suitable for the current scenario, avoiding response delays caused by complex calculations. Through output text encoding and device identification, the generated control commands can accurately specify the guidance content and distribution scope (e.g., only disseminating guidance information in the affected upstream area to avoid interference with unrelated areas), ensuring the effectiveness and accuracy of the guidance information. This logical reasoning method balances decision-making speed and scenario adaptability, guiding traffic flow diversion quickly after an abnormal event occurs, alleviating traffic pressure at the incident point.

[0061] Similarly, based on the aforementioned collaborative control strategy generation, as a further optional implementation method in this embodiment, the output of control commands includes: When the cooperative control strategy includes a cooperative signal timing scheme, the decision execution module inputs the cooperative control strategy into the traffic flow simulator based on the cellular transmission model to obtain the simulated total regional delay. The decision execution module compares the simulated total regional delay with the current total regional delay. Only when the simulated total regional delay is less than the current total regional delay will the cooperative signal timing scheme be issued as a device control command to the traffic control equipment. When the collaborative control strategy includes the display content and publishing strategy of the traffic guidance screen, the display content and publishing strategy of the traffic guidance screen are sent to the traffic control equipment as equipment control commands.

[0062] In practical implementation, the cellular transmission model divides the road network into cells with lengths of 50-100 meters, and uses the flow-density relationship (… ,in For free flow velocity, For the current density, Where w is the maximum flow rate and w is the wave velocity. To simulate traffic flow evolution (using congestion density), the total regional delay is the sum of the waiting times of all vehicles within a cell.

[0063] Secondly, the current total delay in the region Calculated using real-time data from roadside sensors , This refers to the actual travel time. This refers to the time for free circulation.

[0064] Furthermore, since the induction strategy is generated based on historically validated rules and its impact is controllable, simulation verification is not required, and it can be executed directly to ensure timeliness.

[0065] Based on the above, the intelligent traffic monitoring system based on multi-source data fusion in this embodiment introduces a cellular transmission model for simulation verification of the cooperative signal timing scheme. This allows for prediction of the strategy's effectiveness before actual execution: if the simulation shows that the strategy will increase the overall regional delay (e.g., unreasonable green light extensions leading to oversaturation at an intersection), the strategy is rejected to prevent strategy failure or exacerbation of congestion; the strategy is only executed when the simulation confirms that it can reduce delay, ensuring the safety and effectiveness of signal timing adjustments. Traffic guidance strategies, based on mature rules and requiring rapid response, can be directly issued to reduce decision-making delays and ensure timely delivery of guidance information to drivers. This differentiated execution mechanism ensures both strategy effectiveness and response speed, enabling the cooperative control strategy to accurately optimize traffic flow and quickly respond to abnormal events.

[0066] As an optional implementation method of this embodiment, the generation of the road network state feature tensor includes: The spatiotemporal feature fusion engine maps the floating car data after map matching, the roadside sensor data after spatiotemporal alignment, and the video stream data extracted by the convolutional neural network to the same high-dimensional feature space to obtain high-dimensional features. Based on the real-time coverage and historical accuracy of floating car data, roadside sensor data, and video stream data, calculate the intermodal attention weights. By using intermodal attention weights, the mapped high-dimensional features are weighted, concatenated, and nonlinearly transformed to generate a road network state feature tensor.

[0067] In practice, high-dimensional features are mapped using three independent autoencoders: a floating car data autoencoder (inputting 5-dimensional features such as speed and density, and outputting 64-dimensional high-dimensional features), a sensor data autoencoder (inputting 4-dimensional features such as traffic flow and occupancy, and outputting 64-dimensional high-dimensional features), and a video feature autoencoder (inputting 6-dimensional visual features such as vehicle count and queue length, and outputting 64-dimensional high-dimensional features), ensuring that the mapped features have consistent dimensions and similar distributions.

[0068] Secondly, the formula for calculating the intermodal attention weights is: in, Attention weights for the m-th modality (m=1 for floating car, m=2 for sensor, m=3 for video). ); Real-time coverage (the proportion of the road network covered by this modality data, ranging from 0 to 1). Historical accuracy (the average deviation rate between the modal data and the true value, ranging from 0 to 1; the smaller the deviation, the higher the accuracy). The larger); This is the balancing factor (value 0.6, emphasizing the priority of coverage). The real-time coverage rate of the nth modality (n=1,2,3, corresponding to floating car data, roadside sensor data, and video stream data respectively) ranges from [0,1], where 0 indicates that the data of this modality does not cover any road network area, and 1 indicates that it covers the entire target road network area; The historical accuracy rate for the nth mode is defined in the range [0,1]. It is calculated by the average deviation rate between the historical data of this mode and the ground truth traffic data (such as manual verification results or high-precision detector data). The smaller the deviation, the higher the accuracy rate. The larger the value (for example, if the average deviation of a modal data from the true value is 10%), then... ).

[0069] In addition, the weighted splicing method is as follows: ( ( , , (where is the high-dimensional feature of the m-th modality), the nonlinear transformation uses a residual network (the residual network contains 2 convolutional layers and 1 skip connection), and the final output dimension is . The tensor (T is the time step, N is the number of nodes).

[0070] Based on the above, the intelligent traffic monitoring system based on multi-source data fusion in this embodiment maps multimodal data to the same high-dimensional space through an autoencoder, solving the problem of inconsistent feature distribution of heterogeneous data and laying the foundation for subsequent fusion. The calculation of intermodal attention weights comprehensively considers the real-time coverage of the data (e.g., the weight is reduced if the coverage of floating car data is low at night) and historical accuracy (e.g., the accuracy and weight are reduced if the video of a certain road segment is easily affected by weather), enabling the fusion process to dynamically highlight the contribution of high-quality, high-coverage data sources and avoid interference from low-quality data. Weighted splicing and nonlinear transformation further integrate multimodal information, and the generated road network state feature tensor can not only retain the unique value of each modality (e.g., visual details of video, traffic accuracy of sensors), but also form a unified and comprehensive road network state representation. Compared with simple feature splicing or fixed weighted fusion, this method improves the effectiveness of multi-source data fusion and provides a reliable feature foundation for subsequent accurate event recognition and decision-making.

[0071] This embodiment provides a second aspect of a smart traffic monitoring method based on multi-source data fusion, applied to the smart traffic monitoring system based on multi-source data fusion as described above, such as... Figure 2 As shown, the method includes: The data acquisition and fusion stage includes: collecting raw data of various traffic elements in the traffic network; performing fusion processing on the raw data, including map matching of floating car data, spatiotemporal alignment of roadside sensor data, and convolutional neural network feature extraction of video stream data; and performing multimodal fusion on the processed multi-source data to generate a road network state feature tensor under a unified spatiotemporal reference. The hierarchical decision-making and strategy generation stage includes: analyzing the road network state feature tensor through the event perception layer to identify abnormal traffic events; and generating a collaborative control strategy for regional traffic flow based on the identified abnormal traffic events through the regional optimization layer. The strategy execution and feedback optimization phase includes: executing the collaborative control strategy and outputting corresponding equipment control commands to traffic control equipment; collecting historical operational data, including historical road network state feature tensors, historical traffic anomaly event identification results, historical collaborative control strategies, and traffic state change data after execution; and optimizing the decision logic of hierarchical decision-making through online learning based on historical operational data.

[0072] It should be noted that the intelligent traffic monitoring method based on multi-source data fusion in this embodiment corresponds to the aforementioned intelligent traffic monitoring system based on multi-source data fusion. Therefore, the parts of the intelligent traffic monitoring method based on multi-source data fusion in this embodiment that are not specifically described (including but not limited to specific technical means and technical effects) can be referred to the relevant descriptions in the aforementioned intelligent traffic monitoring system based on multi-source data fusion, and will not be repeated here.

[0073] In the embodiments provided by this invention, it should be understood that the embodiments described herein can be implemented in hardware, software, firmware, middleware, code, or any suitable combination thereof. For hardware implementation, the processor can be implemented in one or more of the following: application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), processors, controllers, microcontrollers, microprocessors, other electronic units designed to implement the functions described herein, or combinations thereof. For software implementation, some or all of the processes of the embodiments can be performed by a computer program instructing the associated hardware. During implementation, the program can be stored in a computer-readable storage medium or transmitted as one or more instructions or code on a computer-readable storage medium. Computer-readable storage media include computer storage media and communication media, wherein communication media include any medium that facilitates the transmission of a computer program from one place to another. Storage media can be any available medium accessible to a computer. Computer-readable storage media can include, but are not limited to, RAM, ROM, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage media or other magnetic storage devices, or any other medium capable of carrying or storing desired program code having the form of instructions or data structures and accessible to a computer.

[0074] Finally, it should be noted that the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or make equivalent substitutions for some of the technical features. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A smart traffic monitoring system based on multi-source data fusion, characterized in that, The system includes: The multi-source data acquisition module is used to collect raw data of various traffic elements in the traffic network; A spatiotemporal feature fusion engine is used to perform fusion processing on the original data. The fusion processing includes map matching of floating car data, spatiotemporal alignment of roadside sensor data, and convolutional neural network feature extraction of video stream data to obtain processed multi-source data. The multi-source data is then fused in a multimodal manner to generate a road network state feature tensor under a unified spatiotemporal reference. The hierarchical decision-making core includes an event perception layer and a regional optimization layer; the event perception layer is used to analyze the road network state feature tensor to identify abnormal traffic events; the regional optimization layer is used to generate a collaborative control strategy for regional traffic flow based on the identification results of the abnormal traffic events. The decision execution module is used to execute the collaborative control strategy and output device control commands to the traffic control equipment; The online learning module is used to optimize the decision logic of the hierarchical decision core through online learning based on historical operational data. The historical operational data includes historical road network state feature tensors, historical traffic anomaly event identification results, historical collaborative control strategies, and traffic state change data after the implementation of collaborative control strategies.

2. The intelligent traffic monitoring system based on multi-source data fusion according to claim 1, characterized in that, The identification of the traffic anomaly events includes: The event perception layer inputs the road network state feature tensor and its corresponding road network topology into a preset spatiotemporal graph convolutional network, and the spatiotemporal graph convolutional network outputs the identification result of traffic anomaly events after performing operations on the input data.

3. The intelligent traffic monitoring system based on multi-source data fusion according to claim 2, characterized in that, The operation performed on the input data includes: Perform a graph convolution operation on the input road network state feature tensor and the road network topology to obtain a node feature matrix representing the spatial dependence of the road network; The node feature matrix is ​​input into a gated recurrent unit network to obtain a node state sequence with time dynamics. The node state sequence is input into a fully connected classification layer to obtain the classification results of traffic anomaly events for each node's corresponding road segment.

4. The intelligent traffic monitoring system based on multi-source data fusion according to claim 3, characterized in that, The spatiotemporal graph convolutional network is equipped with a dynamic graph structure update mechanism, which includes: Based on real-time traffic flow data, calculate the weights of the connecting edges between adjacent nodes in the road network topology; The road network topology is dynamically updated based on the calculated weights. The graph convolution operation is performed using the updated road network topology.

5. The intelligent traffic monitoring system based on multi-source data fusion according to claim 1, characterized in that, The generation of the collaborative control strategy includes: The regional optimization layer uses the identification results of the traffic anomaly events as the global state, and fuses them with the local observation states of each agent in the preset signal control model. Through policy network calculation, it outputs a coordinated signal timing scheme; and / or, The regional optimization layer will generate the display content and release strategy of the traffic guidance screen based on the identification results of the traffic anomaly event and the real-time traffic flow data, through logical reasoning using a pre-set decision tree model. The coordinated signal timing scheme and / or the display content and publishing strategy of the traffic guidance screen shall be used as the coordinated control strategy.

6. The intelligent traffic monitoring system based on multi-source data fusion according to claim 5, characterized in that, The policy network computation includes: Each agent concatenates its local observation state with the global state to obtain the agent's comprehensive state vector; The integrated state vector is input into the Actor policy network of the agent to obtain the probability distribution of signal phase actions; Based on the probability distribution, the switching action of the signal phase or the adjustment action of the green light duration are selected and output to form the coordinated signal timing scheme.

7. The intelligent traffic monitoring system based on multi-source data fusion according to claim 5, characterized in that, The logical reasoning includes: The type, severity level, and real-time traffic flow of the upstream road segment of the traffic anomaly are used as input features and fed into the decision tree model. The decision tree model matches predefined rule branches based on the input features and outputs the corresponding guidance screen text code and the target guidance screen device identifier. Based on the text encoding of the guidance screen and the device identifier of the target guidance screen, a traffic guidance screen control command with display content and publishing range is generated to constitute the display content and publishing strategy of the traffic guidance screen.

8. The intelligent traffic monitoring system based on multi-source data fusion according to claim 5, characterized in that, The output of the control command includes: When the cooperative control strategy includes a cooperative signal timing scheme, the decision execution module inputs the cooperative control strategy into a traffic flow simulator based on a cellular transmission model to obtain the simulated total regional delay. The decision execution module compares the simulated total regional delay with the current total regional delay, and only when the simulated total regional delay is less than the current total regional delay is the cooperative signal timing scheme issued as a device control command to the traffic control device. When the collaborative control strategy includes the display content and publishing strategy of the traffic guidance screen, the display content and publishing strategy of the traffic guidance screen are sent to the traffic control device as a device control command.

9. The intelligent traffic monitoring system based on multi-source data fusion according to claim 1, characterized in that, The generation of the road network state feature tensor includes: The spatiotemporal feature fusion engine maps the floating car data after map matching, the roadside sensor data after spatiotemporal alignment, and the video stream data extracted by the convolutional neural network to the same high-dimensional feature space to obtain high-dimensional features. Based on the real-time coverage and historical accuracy of the floating car data, the roadside sensor data, and the video stream data, calculate the intermodal attention weights; The high-dimensional features after mapping are weighted and nonlinearly transformed by the intermodal attention weights to generate the road network state feature tensor.

10. A method for use in an intelligent traffic monitoring system based on multi-source data fusion as described in any one of claims 1-9, characterized in that, The method includes: The data acquisition and fusion stage includes: acquiring raw data of various traffic elements in the traffic network; performing fusion processing on the raw data, including map matching of floating car data, spatiotemporal alignment of roadside sensor data, and convolutional neural network feature extraction of video stream data; and performing multimodal fusion on the processed multi-source data to generate a road network state feature tensor under a unified spatiotemporal reference. The hierarchical decision-making and strategy generation stage includes: analyzing the road network state feature tensor through the event perception layer to identify abnormal traffic events; and generating a collaborative control strategy for regional traffic flow based on the identified abnormal traffic events through the regional optimization layer. The strategy execution and feedback optimization phase includes: executing the collaborative control strategy and outputting corresponding equipment control commands to the traffic control equipment; collecting historical operating data, including historical road network state feature tensors, historical traffic anomaly event identification results, historical collaborative control strategies, and traffic state change data after execution; and optimizing the decision logic of the hierarchical decision-making based on the historical operating data through online learning.

Images