Space-time element association and security demand driven intelligent transportation scene recognition method

Abstract

The application discloses a kind of intelligent traffic scene identification method of space-time element association and security demand driving, comprising: the real-time data of multiple sensors is collected and is synchronized with standardization processing;Multi-dimensional space-time data fusion is implemented to the data after synchronization and standardization processing, and unified space-time data representation is obtained;According to the unified space-time data representation, the safety needs in traffic scene are identified;Using the unified space-time data representation, the semantic analysis of traffic scene is carried out;According to the unified space-time data representation, the safety needs and traffic scene semantic analysis result, traffic management decision is generated.

Description

Technical Field

[0001] This invention relates to the field of intelligent traffic management technology, specifically to an intelligent traffic scene recognition method driven by spatiotemporal element correlation and safety requirements. Background Technology

[0002] With rapid urbanization, the complexity of traffic management has increased significantly, making the safe and efficient guidance of vehicles and pedestrians a major challenge for urban traffic management. Existing Intelligent Transportation Systems (ITS) utilize various sensors, such as cameras, radar, and geomagnetic sensors, to collect data on traffic flow, vehicle speed, and accident information. This data is then analyzed through a central processing system to achieve functions such as traffic flow control, accident warning, and route planning. However, existing technologies often face the problem of data silos, as data collected from various sensors is difficult to integrate effectively in real time, resulting in limited understanding and response capabilities to complex traffic scenarios. Furthermore, traditional ITS is inefficient at processing high-dimensional spatiotemporal data, struggling to capture the rapidly changing spatiotemporal dynamics of traffic flow and the evolving safety requirements in different scenarios. For example, existing systems often cannot quickly adjust to situations where emergency vehicles require priority passage. These limitations not only affect the overall efficiency of the traffic system but also increase safety hazards. Summary of the Invention

[0003] In view of this, the present invention proposes an intelligent traffic scene recognition method driven by spatiotemporal element correlation and safety requirements. By effectively fusing and analyzing data from multiple sensors, it solves the technical problem that existing intelligent traffic systems cannot make real-time and accurate recognition and response to complex traffic scenes.

[0004] To solve the above-mentioned technical problems, the present invention proposes the following technical solution:

[0005] A method for intelligent traffic scene recognition driven by spatiotemporal element correlation and safety requirements includes: synchronizing and standardizing real-time data collected by multiple sensors; performing multi-dimensional spatiotemporal data fusion on the synchronized and standardized data to obtain a unified spatiotemporal data representation; identifying safety requirements in the traffic scene based on the unified spatiotemporal data representation; performing semantic analysis of the traffic scene using the unified spatiotemporal data representation; and generating traffic management decisions based on the unified spatiotemporal data representation, the safety requirements, and the semantic analysis results of the traffic scene.

[0006] The beneficial effects of the above-mentioned technical solution of this invention are reflected in the following: Complex and multi-dimensional data collected by various types of sensors, after synchronization and standardization processing, and multi-dimensional spatiotemporal data fusion, can effectively integrate data from different sensors and time dimensions, overcoming the problem of data silos. Furthermore, it can deeply analyze the inherent connections between spatiotemporal data, identify scenarios driven by safety needs, and adaptively optimize traffic control strategies in real time, thereby improving the accuracy and response speed in complex traffic scenarios. Especially in high-density traffic flow and rapidly changing urban environments, the accurate interpretation and immediate feedback capabilities of spatiotemporal dynamics further highlight the advantages of this invention.

[0007] Furthermore, the synchronization and standardization of real-time data collected by multiple sensors includes: integrating the data collected by multiple sensors within a common time window to achieve data synchronization; scaling, denoising, and timestamp alignment of the synchronized data; wherein, the scaling involves scaling the data to a standard normal distribution, resulting in data with zero mean and unit variance. This is beneficial for subsequent data processing and analysis.

[0008] Furthermore, the multi-dimensional spatiotemporal data fusion of the synchronized and standardized data includes: dynamically encoding data from different sensors to obtain encoded features; integrating the encoded features of multiple sensors into a dynamic spatiotemporal graph network, capturing spatial data relationships through the graph structure, and simultaneously processing time-series data using a dynamic update mechanism to achieve spatiotemporal data fusion. In this further technical solution, the dynamic spatiotemporal graph can simultaneously consider the temporal continuity and spatial relationships of the data, thereby achieving efficient spatiotemporal data fusion and improving the processing efficiency of multi-dimensional data.

[0009] Furthermore, based on the unified spatiotemporal data representation, safety requirements in traffic scenarios are identified, including: defining safety indicators based on a vehicle following model; calculating the values ​​of the safety indicators in real time based on the unified spatiotemporal data representation; identifying safety risks based on the values ​​of the safety indicators; and making corresponding traffic management strategies in response to the identified safety risks.

[0010] Furthermore, the intelligent traffic scene recognition method further includes: continuously learning and optimizing the identification and response strategies for safety needs from real-time data collected by sensors through deep reinforcement learning. Even further, the deep reinforcement learning steps include: constructing a deep reinforcement learning model, which includes system state, actions, rewards, and policies; designing an objective function to maximize long-term cumulative rewards, and designing a loss function to measure the deviation between the current policy and the optimal policy, guiding the optimization and updating of model parameters; and using gradient descent to optimize model parameters to minimize the loss function and improve long-term cumulative rewards. Through deep reinforcement learning, the system continuously learns and optimizes itself, constantly improving the quality of decision-making.

[0011] Furthermore, using the unified spatiotemporal data representation, semantic parsing of traffic scenes is performed, including: constructing a traffic element graph with traffic elements as nodes and interactions between elements as edges; wherein the weight of the edges is determined based on the distance, relative speed, or interaction intensity between elements; assigning different weights to different types of nodes and edges by analyzing traffic patterns and historical traffic accident data; identifying dynamic subgraphs representing preset traffic scenes and communities composed of groups of vehicles with similar behaviors or in similar traffic states from the traffic element graph; mapping the identified dynamic subgraphs and communities to predetermined traffic scene labels according to preset rules; and automatically identifying and labeling scenes using a pre-trained machine learning classifier based on the structural features of the traffic element graph. In this further technical solution, by using the extracted and processed traffic data and constructing a traffic element graph, the complex relationships and dynamic changes between traffic elements in the traffic scene are revealed, thereby achieving deeper semantic parsing.

[0012] Furthermore, based on the unified spatiotemporal data representation, the safety requirements, and the semantic analysis results of traffic scenarios, traffic management decisions are generated, including: constructing a mapping rule knowledge base; identifying traffic conditions in real time based on the unified spatiotemporal data representation; querying matching response measures in the mapping rule knowledge base using a query algorithm based on the real-time identified traffic conditions, the safety requirements, and the semantic analysis results of traffic scenarios; and issuing execution instructions corresponding to the matching response measures to traffic control equipment through a traffic control interface. Further, constructing the mapping rule knowledge base includes: defining a set of characteristic indicators and clarifying a set of management strategies for each traffic condition, while labeling the applicable conditions, expected effects, and priorities of the management strategies; combining existing traffic engineering knowledge and historical traffic data, using machine learning algorithms to analyze management strategies under different traffic conditions, and evaluating and optimizing the effectiveness of the management strategies to improve the accuracy and adaptability of the rules in the knowledge base. A highly optimized mapping rule knowledge base is used to quickly match traffic states and appropriate response measures, achieving immediacy and accuracy in decision-making.

[0013] Furthermore, the intelligent traffic scene recognition method also includes: using a dynamic model based on traffic flow theory to assess and predict the safety status of each traffic node, and deciding whether to activate corresponding safety measures based on the safety status. Detailed Implementation

[0014] The present invention will be further described below with reference to specific implementation methods and embodiments.

[0015] To address the shortcomings of existing intelligent transportation systems in areas such as real-time data fusion, safety-driven scene recognition, adaptive learning, and multimodal data processing, this invention provides a more efficient and secure intelligent traffic management solution, particularly suitable for the ever-increasing urban traffic demands. The intelligent traffic management solution provided by this invention is embodied in two aspects: firstly, it proposes an intelligent traffic scene recognition method based on spatiotemporal element correlation and safety requirements; secondly, it provides a corresponding intelligent traffic scene recognition system. The following detailed description of the steps involved in the proposed intelligent traffic scene recognition method based on spatiotemporal element correlation and safety requirements will also reveal the composition, working principle, and functions of the corresponding intelligent traffic scene recognition system.

[0016] In a specific implementation, the intelligent transportation scene recognition method driven by spatiotemporal element correlation and safety requirements can be described by the following nine steps. It should be understood that these nine steps are not necessarily in chronological order.

[0017] 1. Data Acquisition and Preprocessing

[0018] Data acquisition and preprocessing serve as the foundation, ensuring that multi-source data from different sensors are standardized and synchronized, overcoming the problems of data silos and integration of data in different formats, and laying a solid foundation for subsequent data fusion.

[0019] In intelligent transportation scenarios, multimodal sensor networks need to be deployed to collect multi-dimensional data such as video, radar, and sound. Then, the data undergoes synchronization and standardization to ensure efficient fusion of data from different sources and types, establishing a unified data format for subsequent data fusion, scene analysis, and recognition. This process mainly includes the following parts:

[0020] 1.1 Data Acquisition and Synchronization

[0021] Multimodal sensor networks primarily consist of three types of sensors: video cameras, radar, and sound sensors, each with different data acquisition frequencies and formats. Video cameras provide a continuous stream of images, radar provides distance and velocity measurements, and sound sensors provide noise levels and sound source localization.

[0022] Data synchronization formula:

[0023]

[0024] In equation (1), Sync() is the synchronization function, and D cam D rad D mic These represent the data streams formed after the data from the video camera, radar, and sound sensor are synchronized, respectively. cam (t), d rad (t), d mic (t) represents the video image data, radar data, and audio data at time t, respectively, where t0 and t1 are the start and end times of the data observation window. By integrating these three types of data within a common time window, it can be ensured that the data collected by different sensors are synchronized in time.

[0025] 1.2 Data Standardization Processing

[0026] To ensure data consistency and comparability, all sensor data needs to be converted to a standardized format. In this embodiment of the invention, standardization processes include data scaling, noise reduction, and timestamp alignment.

[0027] Data scaling can be achieved using the following formula (2):

[0028]

[0029] In equation (2), D scaledThis is the scaled data, where D is the original data (in this embodiment, it represents the data after data synchronization as described in section 1.1), and μ... D σ D Let be the mean and standard deviation of the data, respectively. By using formula (2), the data D is scaled to a standard normal distribution, so that the data has zero mean and unit variance, which is beneficial for subsequent data processing and analysis.

[0030] Denoising: First, identify the noise components in the data, which may come from sensor misreading, environmental interference, etc. Then, apply various denoising algorithms, such as filters (e.g., low-pass filters, Kalman filters, etc.), or more advanced techniques, such as wavelet transform denoising, to remove this noise.

[0031] Timestamp alignment: All sensor data must be aligned to a unified time base, precisely matched on the timeline. This step is accomplished through interpolation and resampling, ensuring that data from each sensor can be matched with data from other sensors. In one specific implementation, the following data interpolation formula is used:

[0032]

[0033] In equation (3), D aligned (t) represents the data value calculated using the linear interpolation method at the new alignment time point t, where t i and t i+1 These are adjacent sampling time points, D(t) i ) and D(t i+1 The data values ​​at these sampling time points are (e.g., traffic density or number of vehicles recorded by video cameras; vehicle speed, vehicle distance, or noise level recorded by radar or sound sensors). This interpolation formula, based on the linear assumption, estimates the data value at a new time point t between the data values ​​of two known sampling points to achieve temporal alignment. Formula (3) allows data to be analyzed and compared within a unified time frame, even if the original data have different sampling frequencies.

[0034] 2. Multidimensional spatiotemporal data fusion

[0035] In this embodiment of the invention, multidimensional spatiotemporal data fusion combines data streams (video images, radar data, audio data, etc.) from different sensors after data synchronization and standardization processing to form a unified spatiotemporal data representation, which is crucial for intelligent transportation systems. Specifically, it includes:

[0036] 2.1 Spatiotemporal Data Encoding

[0037] First, dynamic encoding is performed on the data from each sensor. Dynamic encoding involves time series analysis, mapping time-varying data to a stable feature space. In one specific embodiment of the invention, the following dynamic encoding formula can be used:

[0038]

[0039] Where E(t) is the encoding of the sensor data S(t) at the current time t; The decay factor, between 0 and 1, controls the degree of influence of historical data on the current encoding, balancing the impact of the latest data with historical trends; Δt is the time interval, and E(t-Δt) represents the encoding at the previous moment. This formula reflects the smooth transition of data over time and allows the feature space to be dynamically updated over time.

[0040] 2.2 Multidimensional Feature Fusion

[0041] Next, multi-dimensional feature fusion is performed, integrating features encoded by various sensors into a common framework to more accurately represent traffic scenarios. To meet the needs of multi-dimensional feature fusion, especially in processing both temporal and spatial data (temporal data typically refers to time-related variables, indicating changes in parameters such as traffic flow, vehicle speed, and accident occurrence over time; spatial data involves location and layout information, such as road networks, intersection locations, number of lanes, and vehicle positions within the road network), this invention proposes an innovative graph neural network (GNN): Dynamic Spatio-Temporal Graph Network (DST-GN). DST-GN aims to capture the relationships between spatial data through a graph structure while utilizing a dynamic update mechanism to process time-series data, thereby achieving efficient spatio-temporal data fusion.

[0042] In some specific embodiments of the present invention, the steps of spatial data processing and temporal data processing in the dynamic spatiotemporal graph network are as follows:

[0043] ① Spatial data processing

[0044] First, a graph network G = (V, E) is constructed, where V is the set of nodes, representing different data sources (such as data from various sensor observation points); E is the set of edges, representing the spatial relationships between nodes. Spatial data fusion relies on updating node features, which is achieved by aggregating information from neighboring nodes. The node update formula is as follows:

[0045]

[0046] In equation (4), H i (l+1)σ is the feature representation of node i in layer l+1, σ is the activation function, and W (l) B (l) These are weights and biases, which are randomly initialized and then updated and learned according to the node update formula; It is the feature representation of node j, a neighbor of node i, at layer l. It is the set of neighboring nodes of node i. AGGREGATE (l) It is a collection function responsible for collecting and combining the neighbor information of a node to update the representation of each node. Specifically, this function can be summation, averaging, max pooling, etc., with the aim of combining the information of multiple neighbors of a node into a single vector for use as input to the next layer or as the final output.

[0047] ②Time data processing

[0048] The processing of time series data is achieved by introducing a dynamic update mechanism into the graph network. For each time step t, the features of the graph nodes are updated based on the new sensor data. In some specific embodiments of this invention, the following dynamic feature update formula is adopted:

[0049]

[0050] In equation (5), It is the feature representation of node i at time t. It represents the new sensor data received by node i at time t, where U, V, and b are parameters of the update mechanism.

[0051] In this embodiment of the invention, the Dynamic Spatiotemporal Graph Network (DST-GN) graphically represents the spatial data relationships within an intelligent transportation system. Simultaneously, it processes time-series data through a dynamic update mechanism, effectively fusing information from multiple sensors. This spatiotemporal fusion method is particularly suitable for handling the complexities of intelligent transportation scenarios, such as dynamic changes in traffic flow and interactions between different locations.

[0052] By leveraging dynamic spatiotemporal graphs to capture spatial relationships and employing dynamic update mechanisms to process time-series data, efficient spatiotemporal data fusion is achieved, improving the processing efficiency of multi-dimensional data. This approach is particularly suitable for handling the complexities of intelligent transportation scenarios, such as dynamic changes in traffic flow and interactions between different locations. This dynamic spatiotemporal graph, through its structure, simultaneously considers the temporal continuity and spatial relationships of the data. The output not only reflects the situation at a specific moment but also integrates temporal trends and dynamic spatial relationships. Temporal and spatial information is integrated into the final feature representation of each node, supporting a comprehensive understanding of traffic scenarios and decision-making.

[0053] 3. Security requirement identification

[0054] This step aims to identify safety needs in traffic scenarios, such as emergency passage needs and high-risk areas, based on the unified spatiotemporal data representation obtained after the fusion of multidimensional spatiotemporal data. This will facilitate prioritizing traffic scenarios directly related to safety needs in subsequent decision-making.

[0055] First, an improved vehicle car-following model is adopted. This model not only considers the current state of the vehicle but also, by introducing safety indicators as an assessment tool, can predict potential safety issues and identify safety needs. The improved vehicle car-following model is as follows:

[0056]

[0057] In equation (6), v i s is the speed of vehicle i. i It is the actual distance between vehicle i and the vehicle in front, v opt and s opt These represent the optimal speed and optimal distance under predetermined traffic conditions, respectively; α and β are model parameters that reflect the sensitivity to speed and distance adjustments.

[0058] In some specific embodiments of the present invention, two safety indicators are defined: the speed stability index SI. v Safety Index (SI) for vehicle distance s .

[0059] Among them, the speed stability index SI v This value characterizes the smoothness of changes in velocity. Reducing this value can decrease the risk of accidents caused by rapid acceleration or deceleration. The mathematical expression is as follows:

[0060]

[0061] In equation (7), v is the average speed of all vehicles, and N is the total number of vehicles.

[0062] Vehicle Distance Safety Index (SI) s Assessing whether a safe following distance is maintained to avoid a collision can be mathematically represented as follows:

[0063]

[0064] In equation (8), the optimal vehicle distance s opt This reflects the need to maintain a safe following distance at a given speed.

[0065] Based on the above two indicators, security requirements can be identified through the following steps ① to ④:

[0066] ① Setting thresholds: First, based on historical data and security studies, set thresholds for SI v and SI sSet threshold (θ) v and θ s These thresholds are used to determine whether the current traffic situation may pose a safety risk. For example, above θ... v SI v The value may indicate frequent speed changes, increasing the risk of a collision; while SI s Value higher than θ s This may indicate that the distance between vehicles is too close and the safe distance is insufficient.

[0067] ② Real-time data analysis: The system calculates the SI of all vehicles in the current traffic flow in real time. v and SI s This involves collecting data from various sensors (such as cameras, radar, and sound sensors) and updating the speed v of each vehicle based on real-time traffic conditions. i and distance s i information.

[0068] ③ Safety risk identification: When any vehicle's SI v or SI s When the value exceeds a preset threshold, the system identifies the area where the vehicle is located as a high-risk area. The system will analyze the distribution and development trend of these high-risk areas to predict potential safety risks, such as traffic congestion and possible collision points.

[0069] Systematic analysis of the distribution and development trends of these high-risk areas typically involves the following steps:

[0070] (1) Data aggregation: Collect information on all vehicles identified as high-risk and their locations;

[0071] (2) Trend analysis: Use statistical methods or time series analysis to identify the development trend of these high-risk areas, such as whether there is an increasing trend, and the time and location distribution of the risk occurrence;

[0072] (3) Pattern recognition: Applying machine learning models or pattern recognition techniques to analyze and predict specific types of risk patterns, such as the formation of traffic congestion and hotspots for accidents;

[0073] (4) Spatial analysis: Use Geographic Information System (GIS) tools to visualize risk distribution and analyze the relationship between geographic features and risk events.

[0074] (5) Predictive models: Construct predictive models to predict future risk development. These models may be based on historical data to predict future risk levels.

[0075] ④ Response and Intervention: Based on the identified safety risks, the intelligent transportation system responds by adjusting traffic light timings, issuing safety warnings, suggesting or enforcing speed limits, and implementing traffic management strategies such as route planning, thereby intervening in advance to prevent potential accidents from occurring.

[0076] 4. Adaptive learning

[0077] To improve the system's adaptability to unknown traffic modes / scenarios, this embodiment of the invention also designs an adaptive learning framework. Through deep reinforcement learning, it continuously learns from real-time data collected by sensors and optimizes the identification and response strategies for safety needs, thereby continuously improving the quality of decision-making.

[0078] In some specific embodiments of the present invention, the following deep reinforcement learning model is designed, and the model is defined as follows:

[0079] ①State (S): The current state of the system, including key indicators such as traffic flow, speed, and vehicle density, as well as control information such as traffic light status;

[0080] ②Action (A): Actions that the agent can perform, such as adjusting traffic light timings, publishing traffic information, and suggesting route adjustments;

[0081] ③ Reward (R): Based on the impact of the actions taken on traffic flow, design a reward function to encourage positive effects such as reducing congestion and accidents;

[0082] ④ Strategy (π): The agent selects an action based on the current state, and learns the optimal strategy through deep neural network modeling.

[0083] Design a reward function R(s,a) = α1·ΔTrafficFlow + α2·ΔSafety - α3·ΔTravelTime, where R(s,a) represents the reward when the state is s and the action is a, ΔTrafficFlow, ΔSafety and ΔTravelTime represent the changes in traffic flow, safety and travel time, respectively, and α1, α2 and α3 are weighting factors used to adjust the importance of different objectives.

[0084] In the adaptive learning step, we need to define the objective function and loss function for optimization. Since the main objective of this step is to maximize the cumulative future reward, thereby encouraging the policy to improve the overall traffic system performance, the objective function aims to maximize the long-term cumulative reward. The loss function measures the deviation between the current policy and the optimal policy, guiding the optimization and updating of the model parameters.

[0085] The objective function is as follows:

[0086]

[0087] In equation (9), J(θ) represents the cumulative reward function, which is the sum of discounts on all rewards from time t = 0 to infinity under parameter θ; γ t R(s) represents the discount factor, a value between 0 and 1 used to measure the present value of future rewards. A smaller value indicates that the model is more biased towards short-term rewards. t ,a t |θ) represents the action a to be taken at time t, given the parameter θ, in state st. t The immediate reward obtained is based on the system's objectives, such as reducing accidents and alleviating congestion.

[0088] The loss function is defined as follows:

[0089] L(θ)=(Q * (s,a)-Q(s,a|θ)) 2 (10)

[0090] In equation (10), Q * (s,a) is the optimal expected reward when taking action a in state s, Q(s,a|θ) is the reward value currently estimated by the deep reinforcement learning model, and θ represents the parameters of the deep reinforcement learning model.

[0091] Finally, gradient descent is used to optimize the model parameters θ to minimize the loss function and improve long-term rewards. In some specific embodiments of the invention, the updated model parameters θ' are calculated using the following optimization formula:

[0092]

[0093] In equation (11), ε is the learning rate. It is the gradient of the loss function L(θ) with respect to the model parameters θ.

[0094] 5. Semantic analysis of traffic scenarios

[0095] This step aims to utilize the extracted and processed traffic data features, especially the unified spatiotemporal data representation obtained after the aforementioned multidimensional spatiotemporal data fusion, to further combine with the identified safety requirements to perform deep semantic analysis of traffic scenarios, thereby providing richer and more accurate contextual information for traffic decision-making.

[0096] To address this, this invention introduces a graph-based scene understanding framework (GSUF). This framework aims to leverage extracted and processed traffic data features to construct a graph representation of traffic scenes, enabling deeper semantic analysis. Specifically, the GSUF framework represents traffic scenes by constructing a graph model—the Traffic Element Graph (TEG). Nodes represent traffic elements (such as vehicles, pedestrians, and traffic lights), and edges represent interactions between elements (such as following other vehicles and pedestrians crossing roads). This graph model reveals the complex relationships and dynamic changes within traffic scenes, providing a foundation for in-depth semantic analysis.

[0097] Based on this, the specific steps of semantic parsing in traffic scenarios are as follows:

[0098] 5.1 Constructing the Traffic Element Graph (TEG)

[0099] Node definition: Each node represents a traffic element, such as a vehicle, pedestrian, or traffic signal; the attributes of a node include the element's type, location, speed, and direction.

[0100] Edge definition: An edge represents the interaction or relationship between traffic elements, such as following, crossing, or approaching; the weight of an edge can be determined based on the distance, relative speed, or interaction strength between elements.

[0101] 5.2 Feature Fusion and Weighting

[0102] In TEG, certain traffic elements or relationships may be more critical to understanding the scenario. Therefore, by analyzing traffic patterns and previous traffic accident data, different weights are assigned to different types of nodes and edges to enhance the model's sensitivity to key traffic behaviors and potential risks.

[0103] 5.3 Scene Graph Pattern Recognition

[0104] After the Traffic Element Graph (TEG) is constructed, the goal of Scene Graph Pattern Recognition is to identify structural patterns representing specific traffic scenes from the graph. This step mainly includes:

[0105] ① Frequent pattern mining: Apply graph mining algorithms (such as gSpan) to search for frequently occurring subgraph patterns in TEG. These frequently occurring subgraph patterns may correspond to common traffic scenarios, such as convoy driving or traffic congestion.

[0106] ② Dynamic Subgraph Recognition: Identifying subgraph patterns that dynamically form over time is called dynamic subgraphs. These dynamic subgraphs reflect instantaneous changes in traffic flow, such as congestion caused by sudden traffic accidents. Dynamic subgraph recognition requires tracking the temporal changes of nodes and edges in the TEG, specifically including the following 6 steps:

[0107] (1) Timestamps: Ensure that every node and edge in the TEG has a timestamp that reflects when the data was collected. This is the basis for tracking how elements change over time.

[0108] (2) Window division: Define time windows for observing and analyzing changes in nodes and edges. These windows can be fixed (e.g., every 5 minutes) or based on specific events (e.g., when traffic flow changes suddenly).

[0109] (3) Subgraph Extraction: Within each time window, extract subgraphs that appear frequently or change significantly. Use graph mining algorithms (such as gSpan) to identify these patterns. The focus is on identifying subgraphs that show characteristics such as significantly increased or decreased traffic flow and emergency response.

[0110] (4) Change detection: Analyze the changes in subgraphs within a continuous time window to identify subgraph patterns that change significantly in a short period of time. These changes may indicate the onset of congestion or the occurrence of an accident.

[0111] (5) Mark dynamic subgraphs: Mark subgraphs with significant changes detected as dynamic subgraphs. These subgraphs represent key instantaneous changes in traffic flow, such as congestion caused by accidents or roadblocks.

[0112] (6) Context fusion: Combine other data sources (such as weather conditions and special event information) to analyze the causes and possible impacts of these dynamic subgraphs.

[0113] ③ Community Detection: Community detection algorithms (such as the Louvain method) are used to identify tightly connected sets of nodes in the TEG. These sets of nodes represent groups of vehicles with similar behaviors or in similar traffic conditions, providing important spatial clustering information for traffic management.

[0114] 5.4 Semantic Tag Mapping

[0115] Rule-based mapping: Based on rules developed by traffic engineering experts, identified dynamic subgraphs and communities are mapped to specific traffic scenario labels. For example, a frequently occurring subgraph of densely packed vehicles might be mapped as a "high-risk congestion area".

[0116] Machine learning classification: For complex traffic scenarios, machine learning classifiers (such as random forests and support vector machines) are used to automatically identify and label scenarios based on the structural features of the traffic environment (TEG). Training data can come from historical traffic events and expert-annotated scenarios.

[0117] 5.5 Scenario Understanding and Decision Support

[0118] Traffic management recommendations: Based on scenario labels, the system generates traffic management recommendations, such as signal control adjustments, route guidance, and speed limits. These recommendations aim to improve identified problem scenarios, such as alleviating congestion or enhancing safety in specific areas.

[0119] Real-time feedback loop: After implementing recommendations, the system evaluates the effectiveness of management measures by monitoring their effects (such as changes in traffic flow and reductions in accident rates). Based on this feedback, the system adjusts its identification and mapping rules to optimize future scenario understanding and traffic management decisions.

[0120] 6. Real-time decision-making

[0121] This step aims to generate real-time traffic management decisions based on real-time data, identified safety needs, and semantic analysis results of traffic scenarios. A highly optimized rule knowledge base is used to quickly match traffic conditions with appropriate response measures, achieving both immediacy and accuracy in decision-making. Specifically, this includes:

[0122] 6.1 Construction of Mapping Rule Knowledge Base

[0123] Knowledge base content construction: For each traffic condition (minor congestion, major accident, special event, etc.), define a set of detailed characteristic indicators, such as traffic density, speed reduction rate, accident severity level, etc.; for each traffic condition, define a set of management strategies and mark their applicable conditions, expected effects and priorities. For example, prioritize signal adjustment in minor congestion and prioritize route redirection in major accidents.

[0124] Rule Formulation and Optimization: Combining the knowledge of traffic engineering experts with historical traffic data, machine learning algorithms (such as decision trees and support vector machines) are used to analyze effective management strategies under different traffic conditions. The effectiveness of these management strategies is evaluated and optimized using algorithms to ensure that the rules in the knowledge base are highly accurate and adaptable.

[0125] The process of evaluating and optimizing the effectiveness of management strategies in the mapping rule knowledge base may include:

[0126] (1) Historical data analysis: Use historical traffic data to train and validate predefined management strategies. For example, analyze the historical effects of strategies implemented under specific traffic conditions (such as signal adjustments or route redirections).

[0127] (2) Simulation test: Test the effect of each strategy in a simulated environment and observe the performance and impact of different strategies under similar traffic conditions.

[0128] (3) Algorithm application: Apply machine learning algorithms (such as decision trees and support vector machines) to optimize strategy selection. These algorithms can automatically recommend the most effective strategies based on the input traffic condition features.

[0129] (4) Effect evaluation: Monitor the effects of the strategy after implementation through real-time data, such as improved traffic flow and reduced accident rate.

[0130] (5) Feedback loop: Feedback the implementation effect to the knowledge base and adjust the strategy parameters and applicable conditions according to the actual performance.

[0131] 6.2 Real-time traffic condition processing

[0132] Real-time data reception: Deploy distributed data acquisition nodes (multimodal sensor network) and utilize 5G or other high-speed communication technologies to achieve rapid connection with traffic sensors and cameras, reducing data transmission latency. Furthermore, edge computing technology can be used to perform preliminary analysis at the data source points, such as rapid traffic flow statistics and anomaly detection, reducing the burden on the central processing system.

[0133] These data can be further processed using steps 1 and 2 above. Implement a real-time streaming data processing engine, such as using Apache Flink, and configure a Complex Event Processing (CEP) mode to identify changes in traffic conditions in real time. In some embodiments, the specific steps for identifying changes in traffic conditions include: (1) Data collection: Collect traffic data in real time from multiple sources such as traffic cameras, sensors, and GPS devices; (2) Data preprocessing: Clean and format the collected data to ensure data quality, such as removing outliers and filling missing values; (3) Streaming data processing configuration: Configure the data stream inlet using a streaming processing platform such as Apache Flink, and set an appropriate window size for batch data processing; (4) Complex Event Processing (CEP): Configure the CEP mode in Apache Flink, define and implement rules for detecting specific traffic events (such as sudden traffic stoppage, sharp increase in traffic flow, etc.); (5) Event detection: Analyze the data stream in real time, match patterns according to predefined rules, and identify events that indicate changes in traffic conditions; (6) Response triggering: Once a critical event is detected, the system automatically triggers an alarm or notifies relevant departments to take action.

[0134] Machine learning models are introduced to analyze traffic data flow in real time, such as models trained on real-time traffic data to quickly predict congestion trends. This process specifically includes: (1) Model selection: Selecting a suitable machine learning model, such as random forest, support vector machine or neural network, to predict traffic trends; (2) Feature engineering: Constructing features based on real-time traffic data, which may include traffic flow, speed, time, weather conditions, etc.; (3) Model training: Training the prediction model using historical traffic data, which may involve adjusting model parameters, performing cross-validation and other optimization operations; (4) Real-time model update: Periodically updating the model to include the latest traffic conditions as new data is continuously added; (5) Prediction execution: The model receives new traffic data in real time, makes rapid predictions, and outputs congestion trends and possible developments; (6) Decision support: Providing the prediction results to the traffic management center to help formulate traffic control or guidance strategies.

[0135] 6.3 Decision Mapping and Execution

[0136] Status and Strategy Mapping: Based on the real-time traffic conditions identified in section 6.2, a matching response measure is quickly found in the mapping rule knowledge base using efficient query algorithms (such as hash lookup and B-tree).

[0137] Preferably, it also includes optimizing the knowledge base storage and index structure to ensure millisecond-level query response times even with large-scale data and complex rule sets.

[0138] Decision execution: Design an event-driven response framework that automatically generates execution instructions and quickly issues them to the corresponding traffic control equipment through an integrated traffic control interface when a matching management strategy is selected.

[0139] In addition, a real-time monitoring and logging mechanism is implemented to ensure that the execution status of each decision can be tracked and analyzed in real time.

[0140] 7. Security Response Coordination Mechanism

[0141] A dynamic model based on traffic flow theory is used to assess and predict the safety status of each traffic node, and the activation of corresponding safety measures is determined based on the safety status to ensure the timely and effective implementation of traffic management measures. Specifically, a mechanism is established to translate the identification and decision-making results into concrete traffic control actions. This includes proposing an integrated scheme, called the Dynamic Safety Response System (DSRS), to address the needs of the safety response coordination mechanism. DSRS aims to use algorithms based on physical models and mathematical formulas to assess traffic conditions in real time, predict potential risks, and coordinate the implementation of safety response measures to optimize traffic flow and improve public safety.

[0142] Detailed description of the DSRS scheme:

[0143] ① Dynamic Safety Assessment: DSRS employs a dynamic model based on traffic flow theory to assess and predict the safety status of each traffic node. Considering traffic flow, vehicle speed, and road conditions, a safety index SI is defined to quantify the safety risk of each node.

[0144]

[0145] f(Q,V,C) represents the formula for calculating the safety index, where Q is the traffic flow, V is the average vehicle speed, C is the road condition score, and a, b, and c are weighting coefficients determined by historical data analysis. The road condition score C considers factors such as road surface conditions and visibility, and is derived from professional assessments or sensor data.

[0146] ② Safety Response Strategy Selection and Execution: Based on the real-time calculation results of the Safety Index (SI), DSRS determines whether safety response measures need to be activated using preset thresholds. These measures include, but are not limited to, traffic light adjustments, route redirection, and speed limit announcements.

[0147] ③ Policy Mapping Rules: Based on the SI value, the system will select an appropriate response measure from the policy library. For example, when the SI exceeds a certain high-risk threshold, the system may prioritize initiating emergency route redirection.

[0148] 8. User Interface and Feedback System

[0149] This step aims to provide a user interface and self-feedback system for system performance feedback, enhancing the user experience and providing data for continuous system improvement. These functions are achieved through the following two key components:

[0150] ① Real-time traffic data display

[0151] Data visualization: Utilizing charts, maps, and real-time streaming media to display current traffic conditions to users, including but not limited to traffic flow, speed, congested areas, and accident information. Employing GIS (Geographic Information System) technology to combine traffic data with geographic location data provides users with intuitive spatial information references.

[0152] ② User feedback collection and processing

[0153] Feedback Interface: Provides a clean and user-friendly interface that allows users to submit traffic-related feedback and suggestions via mobile app or website. This includes accident reports, congestion information, and reports of poor road conditions.

[0154] Feedback Processing Flow: Establish an automated feedback processing flow, including initial classification, urgency level assessment, and feedback archiving. For example, use a simple classification algorithm to categorize feedback into categories such as "accident," "congestion," and "road conditions."

[0155] 9. System Integration and Optimization

[0156] This step aims to integrate the various modules and processes into a coordinated whole, ensuring the system works collaboratively and enabling continuous performance evaluation and optimization. This part relies on four core components: modular design, data synchronization and integration, performance monitoring and feedback adjustments, and cross-system collaboration.

[0157] ① Modular design

[0158] Design Principles: A modular design principle is adopted, breaking down the intelligent transportation system into independent but collaborative modules, such as data acquisition, risk assessment, decision-making, and execution. Each module is designed with standardized interfaces to facilitate overall system integration and subsequent expansion or upgrades.

[0159] ② Data synchronization and fusion

[0160] Synchronization Mechanism: Design an efficient data synchronization mechanism to ensure that real-time data collected from various data sources (such as traffic cameras, sensors, and GPS) can be quickly aggregated to the central processing system;

[0161] Fusion algorithms: Applying data fusion algorithms to process aggregated data and optimize data quality, such as improving the accuracy and completeness of data through time series analysis and spatial analysis.

[0162] ③Performance monitoring and feedback adjustment

[0163] KPIs Definition and Monitoring: Define key performance indicators (KPIs), such as response time, system stability, and user satisfaction, and evaluate system performance by monitoring these KPIs in real time;

[0164] Feedback mechanism: Based on monitoring results, the system automatically or manually adjusts and optimizes system configuration and module algorithms to continuously improve system performance.

[0165] ④ Cross-system collaboration

[0166] Collaboration Protocols: Establish standardized communication protocols and data formats to support effective collaboration with other intelligent transportation subsystems (such as emergency response systems and public transportation systems).

[0167] Based on the detailed process steps described above, the key aspects of the intelligent traffic scene recognition method driven by spatiotemporal element correlation and safety requirements proposed in this embodiment of the invention are as follows: First, data acquisition and preprocessing serve as the foundation, ensuring that multi-source data from different sensors are standardized and synchronized, laying a solid foundation for subsequent data fusion; then, the multi-dimensional spatiotemporal data fusion step uses graph neural networks to perform deep structural processing on these data, thereby revealing the spatiotemporal correlations between different data; next, the safety requirement identification step utilizes these correlations to identify key safety requirements in the traffic scene, thereby driving subsequent decision-making; traffic scene semantic parsing, based on the extracted deep meaning in the data, transforms sensor data into useful information that the system can accurately understand and respond to, realizing traffic scene semantic parsing, which provides the necessary input for the real-time decision engine; finally, the decision engine quickly generates response measures based on the previously extracted spatiotemporal data, identified safety requirements, and scene semantic parsing results, in order to maximize the safety and efficiency of traffic flow.

[0168] Furthermore, as a preferred approach, step 4, adaptive learning, improves the system's adaptability to unknown traffic patterns through continuous learning and optimization, ensuring that it can make optimal decisions under various circumstances; step 7, the safety response coordination mechanism, ensures that these decisions can be effectively implemented in the real world, transforming the theoretical advantages of intelligent systems into practical improvements in traffic management; step 8, the user interface and feedback system, not only enhances the user experience but also provides the system with the possibility of continuous improvement by collecting user feedback; and step 9, system integration and optimization, can improve the collaborative efficiency of each part and achieve optimal performance among various modules.

[0169] Another embodiment of the present invention proposes a computer-readable storage medium storing a computer program thereon. When executed by a processor, this computer program can implement the steps of the intelligent traffic scene recognition method described in the foregoing embodiments. Based on this understanding, the technical solution of the aforementioned intelligent traffic scene recognition method can be embodied in the form of a software product. This software product can be stored in a non-volatile storage medium (such as a CD-ROM, USB flash drive, or portable hard drive), including several instructions to cause a computer device (such as a personal computer, server, or network device) to execute the methods of various embodiments of the present invention.

[0170] Based on the foregoing description, the "Spatiotemporal Element Association and Safety Demand-Driven Intelligent Traffic Scene Recognition System (STEA-SRDITSRS)" corresponding to the intelligent traffic scene recognition method in the aforementioned embodiments has the following improved functions compared to traditional intelligent traffic systems:

[0171] 1) Multidimensional spatiotemporal data fusion capability: Through an innovatively designed graph neural network architecture and data fusion algorithm, STEA-SRDITSRS can effectively integrate data from different sensors and time dimensions, overcoming the data silo problem and providing a more comprehensive understanding of traffic scenarios.

[0172] 2) Safety-Driven Scene Recognition: Unlike traditional ITS's passive monitoring of traffic flow, STEA-SRDITSRS introduces a proactive recognition mechanism based on safety requirements. The system can identify and prioritize scenarios with the greatest impact on traffic safety in real time, such as emergency vehicle passage and crowd management for major events.

[0173] 3) Adaptive learning mechanism: The system adopts deep reinforcement learning method to continuously learn and optimize the recognition model and response strategy from real-time data. This enables STEA-SRDITSRS to adapt quickly when encountering unknown or changing traffic patterns, which significantly improves the flexibility and effectiveness of traffic management.

[0174] 4) Multimodal data processing: STEA-SRDITSRS's multimodal processing capabilities enable the system to understand and analyze richer and more complex traffic environment information, including unstructured data (such as real-time traffic reports on social media), providing deeper insights for traffic management.

[0175] 5) Real-time response and prevention mechanism: The system processes and analyzes the results in real time through advanced algorithms, and can automatically adjust traffic control facilities such as traffic lights and information signs when potential risks are detected, thus achieving a fast and accurate safety response.

[0176] The above description, in conjunction with specific preferred embodiments, provides a further detailed explanation of the present invention. It should not be construed that the specific implementation of the present invention is limited to these descriptions. For those skilled in the art, several equivalent substitutions or obvious modifications can be made without departing from the concept of the present invention, and all such modifications, achieving the same performance or purpose, should be considered within the scope of protection of the present invention.

Claims

1. A method for intelligent traffic scene recognition driven by spatiotemporal element correlation and safety requirements, characterized in that, include: Synchronize and standardize real-time data collected by multiple sensors; Multidimensional spatiotemporal data fusion is performed on the synchronized and standardized data to obtain a unified spatiotemporal data representation; The multidimensional spatiotemporal data fusion includes spatiotemporal data encoding and multidimensional feature fusion. Spatiotemporal data encoding involves dynamically encoding data from different sensors to obtain encoded features. Multidimensional feature fusion integrates the encoded features from multiple sensors into a dynamic spatiotemporal graph network. This network captures spatial data relationships through a graph structure and processes time-series data using a dynamic update mechanism, thus achieving spatiotemporal data fusion. The dynamic spatiotemporal graph network captures spatial data relationships through a graph structure and processes time-series data using a dynamic update mechanism, fusing temporal trends and dynamic spatial relationships. Temporal and spatial information is integrated into the final feature representation of each node. Based on the unified spatiotemporal data representation, safety requirements in traffic scenarios are identified; Using the unified spatiotemporal data representation, semantic analysis of traffic scenarios is performed; Traffic management decisions are generated based on the unified spatiotemporal data representation, the security requirements, and the semantic parsing results of the traffic scenario. The unified spatiotemporal data representation is used to perform semantic parsing of traffic scenarios, including: A scene understanding framework based on graph models is introduced, which uses traffic elements as nodes and the interactions between elements as edges to construct a traffic element graph to represent traffic scenes. The nodes represent traffic elements, the edges represent the interactions between elements, and the weights of the edges are determined based on the distance, relative speed, or interaction intensity between elements. By analyzing traffic patterns and historical traffic accident data, different weights are assigned to different types of nodes and edges. Identify dynamic subgraphs representing preset traffic scenarios from the traffic element graph, as well as communities composed of groups of vehicles with similar behaviors or in similar traffic conditions. According to preset rules, the identified dynamic sub-graph and the community are mapped to predetermined traffic scene labels; A pre-trained machine learning classifier is used to automatically identify and label scenes based on the structural features of the traffic element map.

2. The intelligent traffic scene recognition method as described in claim 1, characterized in that, The process of synchronizing and standardizing real-time data collected by multiple sensors includes: Data from multiple sensors is integrated within a common time window to achieve data synchronization. The synchronized data is scaled, denoised, and timestamp aligned; wherein, the scaling is to scale the data to a standard normal distribution, so that the data has zero mean and unit variance.

3. The intelligent traffic scene recognition method as described in claim 1, characterized in that, Based on the unified spatiotemporal data representation, safety requirements in traffic scenarios are identified, including: Based on the vehicle car-following model, safety indicators are defined; The values ​​of the security indicators are calculated in real time based on the unified spatiotemporal data representation. Identify security risks based on the values ​​of the aforementioned security indicators; In response to identified safety risks, appropriate traffic management strategies are implemented.

4. The intelligent traffic scene recognition method as described in claim 1, characterized in that, Also includes: Through deep reinforcement learning, we continuously learn from real-time data collected by sensors and optimize strategies for identifying and responding to security needs.

5. The intelligent traffic scene recognition method as described in claim 4, characterized in that, The steps of the deep reinforcement learning include: Construct a deep reinforcement learning model, which includes system state, action to be performed, reward, and policy; The objective function is designed to maximize the long-term cumulative reward, and a loss function is designed to measure the deviation between the current policy and the optimal policy, thereby guiding the optimization and updating of the model parameters. Gradient descent is used to optimize model parameters in order to minimize the loss function and improve long-term cumulative reward.

6. The intelligent traffic scene recognition method as described in claim 1, characterized in that, Based on the unified spatiotemporal data representation, the security requirements, and the semantic parsing results of the traffic scenario, traffic management decisions are generated, including: Build a mapping rule knowledge base; Traffic conditions are identified in real time based on the unified spatiotemporal data representation. Based on the real-time traffic conditions identified, and based on the safety requirements and the semantic parsing results of the traffic scenario, a query algorithm is used to query the mapping rule knowledge base for matching response measures. The traffic control interface issues execution commands corresponding to the matched response measures to the traffic control equipment.

7. The intelligent traffic scene recognition method as described in claim 6, characterized in that, The construction of the mapping rule knowledge base includes: Define a set of characteristic indicators and a set of management strategies for each traffic condition, and indicate the applicable conditions, expected effects and priorities of the management strategies; By combining existing traffic engineering knowledge and historical traffic data, machine learning algorithms are used to analyze management strategies under different traffic conditions, and the effectiveness of the management strategies is evaluated and optimized to improve the accuracy and adaptability of the rules in the knowledge base.

8. The intelligent traffic scene recognition method as described in claim 1, characterized in that, Also includes: A dynamic model based on traffic flow theory is used to assess and predict the safety status of each traffic node, and to decide whether to activate safety response measures based on the safety status.

Images