Traffic video monitoring image intelligent analysis method based on deep learning

Abstract

The invention relates to the technical field of intelligent traffic monitoring, and discloses a traffic video monitoring image intelligent analysis method based on deep learning. Comprising the following steps: constructing a double-flow heterogeneous graph network by using traffic video streams and geographic information data, and generating multi-dimensional context information containing a driving style and a group mode; calculating the structural tension of the real track in the network, and generating an anti-factual reasoning trigger instruction when the condition is met; in response to the instruction, instantiating the virtual obstacle and generating a personalized anti-fact trajectory; and obtaining a driving intention judgment result based on the trajectory similarity and the multi-dimensional context information, and updating a static rule. According to the method, the double-flow heterogeneous graph network is constructed, the personalized anti-fact trajectory is generated when the structural tension meets the condition, the correlation degree of the deviation behavior and the assumed risk avoiding behavior is quantified according to the trajectory similarity, and the accuracy of traffic behavior analysis is improved on the basis of distinguishing substantive traffic violation and compliance hidden risk avoidance.

Description

Technical Field

[0001] This invention relates to the field of intelligent traffic monitoring technology, specifically to a method for intelligent analysis of traffic video surveillance images based on deep learning. Background Technology

[0002] Current traffic video surveillance technology is widely used in urban traffic management and violation evidence collection. Mainstream solutions typically utilize computer vision algorithms to detect vehicles and track their trajectories in video streams, mapping the extracted vehicle location information to a pre-defined geospatial coordinate system. These systems rely on preset static traffic rules, such as lane dividers, no-entry zones, and directional arrows. By calculating the geometric positional relationship between the vehicle's bounding box or center point and these static rule elements—for example, whether there is spatial overlap or vector crossing—they determine whether a vehicle has committed traffic violations such as running a red light, crossing a lane, or driving in the wrong direction.

[0003] However, existing rule-matching-based analysis methods primarily focus on the physical manifestations of vehicle behavior, lacking the ability to reason about the deeper motivations behind driving behavior. They typically employ a binary judgment logic, classifying a vehicle as a violation simply by its trajectory entering a prohibited area. This makes it difficult to effectively distinguish between a driver's intentional, substantive violation and a compliant evasive maneuver to avoid hidden risks such as potholes or falling objects. Furthermore, the static rule maps used in existing monitoring systems are usually fixed and cannot reflect the collective evasive behavior of vehicles during actual traffic, failing to detect and mark dynamic hazards on the road. This leads to persistent misjudgments when faced with atypical driving behaviors caused by environmental changes. Simultaneously, existing trajectory analysis algorithms often use generic kinematic models, failing to adequately consider the differences in driving styles among different drivers, resulting in a lack of personalized quantitative benchmarks when evaluating the rationality of evasive trajectories in complex scenarios.

[0004] Therefore, how to construct a traffic video surveillance image analysis method that can accurately distinguish between substantive violations and compliant risk-avoidance behaviors and can adaptively update the rule representation based on environmental interaction information is a technical problem that needs to be solved in this field. Summary of the Invention

[0005] To address the shortcomings of existing technologies, this invention provides a deep learning-based intelligent analysis method for traffic video surveillance images, which solves the problems of existing technologies being unable to accurately distinguish between substantive traffic violations and compliance-related hidden risk avoidance behaviors, as well as the inability of static monitoring rules to adapt to dynamic changes in the traffic environment.

[0006] This invention provides a deep learning-based intelligent analysis method for traffic video surveillance images, employing the following technical solution:

[0007] A deep learning-based intelligent analysis method for traffic video surveillance images includes the following steps:

[0008] By utilizing traffic video streams and geographic information data, a dual-stream heterogeneous graph network representing the interaction between dynamic entities and static rules is constructed.

[0009] Based on the dual-stream heterogeneous graph network, multi-dimensional contextual information including driving style and group pattern is generated;

[0010] The structural tension of the vehicle's actual trajectory in the dual-stream heterogeneous graph network is calculated. When the structural tension meets the preset triggering conditions, a counterfactual reasoning triggering command is generated.

[0011] In response to the counterfactual reasoning trigger command, virtual obstacles are instantiated in the dual-stream heterogeneous graph network and combined with the multi-dimensional contextual information to generate personalized counterfactual trajectories;

[0012] Calculate the trajectory similarity between the personalized counterfactual trajectory and the real trajectory, and obtain the driving intention determination result based on the trajectory similarity and the multi-dimensional contextual information;

[0013] The static rules in the dual-stream heterogeneous graph network are updated based on the driving intention determination result to obtain the evolved dual-stream heterogeneous graph network.

[0014] By employing the aforementioned technical solution, this method semantically couples dynamic traffic participants with static road rules using a dual-stream heterogeneous graph network and introduces a counterfactual reasoning mechanism. This changes the traditional monitoring model that relies solely on binary judgment based on rule lines. It measures the degree of conflict between vehicle behavior and rules by calculating structural tension, and simulates counterfactual scenarios under non-violation conditions by constructing virtual obstacles when high conflicts occur. By comparing real trajectories with counterfactual trajectories, the system can infer whether the driver's violation was an attempt to avoid potential risks, i.e., compliance risk avoidance.

[0015] Furthermore, by updating static rules based on the judgment results, the system can achieve self-evolution of monitoring rules and automatically adapt to hidden risks such as obstructed vision or road surface defects, thereby reducing false alarm rates and improving the level of intelligence in traffic law enforcement.

[0016] Furthermore, the construction of the two-stream heterogeneous graph network representing the interaction between dynamic entities and static rules includes:

[0017] The traffic video stream is processed using a target detection model and tracking algorithm, which maps traffic participants as dynamic entity nodes and constructs edges based on the spatial distance between the dynamic entity nodes to form a dynamic perception subgraph.

[0018] The geographic information data is parsed, the static elements of roads are mapped to static rule nodes, and edges are constructed based on physical connection relationships to form a static knowledge subgraph;

[0019] The spatial reference of the dynamic perception subgraph and the static knowledge subgraph is unified by a coordinate transformation matrix to obtain the dual-stream heterogeneous graph network.

[0020] By adopting the above technical solution, this invention solves the problem of data heterogeneity between the video pixel coordinate system and the geospatial coordinate system. The dynamic perception subgraph captures the real-time interactive features of traffic flow, while the static knowledge subgraph provides the topological constraints of roads. The unified spatial reference of the two enables the algorithm to understand the relative relationship between vehicles and rules within the same semantic space, providing an accurate data foundation for subsequent tension calculations.

[0021] Furthermore, the generation of multi-dimensional contextual information including driving style and group pattern includes: extracting dynamic features of the real trajectory and performing cluster analysis to determine the driving style category to which the vehicle belongs, thereby obtaining the driving style; performing correlation analysis on the trajectories of multiple vehicles within a preset spatiotemporal window to identify abnormal trajectory clusters that deviate from the standard path, thereby obtaining the group pattern.

[0022] By adopting the above technical solutions, the system incorporates personalized and group characteristics of driving behavior. Identifying driving styles helps distinguish between intentional violations and passive avoidance in subsequent reasoning; while identifying group patterns can detect large-scale collective detours caused by sudden changes in road conditions, thereby avoiding misjudgments of group risk avoidance behavior.

[0023] Furthermore, the step of generating a counterfactual reasoning trigger command when the structural tension meets a preset trigger condition includes:

[0024] The cumulative depth of the actual trajectory intruding into the prohibited region defined in the static rule is calculated as the structural tension.

[0025] Determine whether the structural tension exceeds a preset threshold, and detect whether there are dynamic entities in the dual-stream heterogeneous graph network that can explain the deviation behavior;

[0026] If the structural tension exceeds the preset threshold and the dynamic entity does not exist, the counterfactual reasoning trigger command is output.

[0027] By adopting the above technical solution, this invention establishes an effective reasoning triggering mechanism. Structural tension quantifies the severity of the violation, while detecting the presence of explanatory dynamic entities excludes explicit avoidance situations caused by sudden stops of vehicles ahead. The system triggers counterfactual reasoning only when a vehicle commits a serious violation and there are no obvious dynamic disturbances in the surrounding area, focusing on identifying abnormal driving caused by implicit risks, thus ensuring detection accuracy and optimizing the allocation of computational resources.

[0028] Furthermore, instantiating virtual obstacles in the dual-stream heterogeneous graph network includes:

[0029] Calculate the maximum curvature point of the deviation segment in the real trajectory; determine the initial coordinates of the virtual obstacle based on the normal direction of the maximum curvature point and its position inside the trajectory; if there are empirical risk anchor points in the dual-stream heterogeneous graph network, correct the initial coordinates using the positions of the empirical risk anchor points to obtain the target coordinates and generate the virtual obstacle.

[0030] By employing the aforementioned technical solutions, the system can infer the location of potential threats that would cause the driver to take evasive action. The point of maximum curvature typically corresponds to the moment when the driver's reaction is most pronounced, thus inferring the location of virtual obstacles in accordance with driving kinematics. Simultaneously, the introduction of historically accumulated empirical risk anchor points to correct the coordinates ensures that the construction of the virtual scene not only relies on the current frame data but also incorporates historical risk priors for that road segment, improving the accuracy of counterfactual scenarios.

[0031] Furthermore, the generation of personalized counterfactual trajectories includes: searching for regression paths using a path planning algorithm in the dual-stream heterogeneous graph network that incorporates the virtual obstacles; constructing a cost function that includes path length, curvature, and safety distance; and dynamically adjusting the weights of the cost function according to the driving style to obtain the personalized counterfactual trajectory.

[0032] By employing the above technical solution, the generated counterfactual trajectory is not a simple geometric connection, but a drivable path that conforms to vehicle dynamics and the specific driver's style. The cost function weights are dynamically adjusted according to driving style to ensure that the generated trajectory reflects the normal reaction a particular driver should have when facing a hypothetical obstacle, thus making the comparison between the real trajectory and the counterfactual trajectory logically comparable.

[0033] Furthermore, obtaining the driving intention determination result includes: calculating the distance between the real trajectory and the personalized counterfactual trajectory using a dynamic time warping algorithm to obtain the trajectory similarity; inputting the trajectory similarity and the multi-dimensional context information into a classification model to calculate the posterior probability of the compliance and risk avoidance category; and outputting the driving intention determination result based on the posterior probability.

[0034] By employing the aforementioned technical solution, the influence of vehicle speed changes on trajectory comparison is overcome using the dynamic time warping algorithm, quantifying the similarity between real-world behavior and theoretical hazard-avoidance behavior. Combined with posterior probability calculations based on contextual information, a mapping from geometric similarity to semantic intent is achieved, accurately determining whether the driver is performing necessary hazard-avoidance maneuvers or committing a substantive traffic violation.

[0035] Furthermore, updating the static rules in the dual-stream heterogeneous graph network based on the driving intention determination result includes: statistically analyzing the frequency of events judged as compliant implicit risk avoidance at a preset geographical location; when the event frequency meets a preset update condition, creating or updating an empirical risk anchor point at the preset geographical location of the static rule. Preferably, the empirical risk anchor point includes a confidence weight attribute, and creating or updating the empirical risk anchor point includes iteratively updating the confidence weight attribute using the determination confidence of the event.

[0036] By adopting the above technical solution, this invention endows the monitoring system with the ability to self-update. Through accumulating multiple compliance and hazard avoidance judgments, the system can identify areas that human drivers generally avoid, which often correspond to physical hazards that are difficult to identify in the monitoring footage. Feeding this implicit knowledge back to the static rule network makes subsequent analysis more accurate, enabling the rules to adapt to environmental changes.

[0037] Furthermore, after updating the static rules in the dual-stream heterogeneous graph network based on the driving intention determination result, the method further includes: when the driving intention determination result is a compliant implicit risk avoidance, generating a road potential hazard warning that includes the location and type of the hazard, and sending the road potential hazard warning to the management terminal.

[0038] By adopting the above technical solutions, the system is not only used for law enforcement but also transformed into an auxiliary tool for road infrastructure maintenance. By detecting abnormal avoidance behaviors of vehicles, the system can reverse-engineer potential hazards such as water accumulation or potholes on the road and promptly notify management departments, demonstrating the value of intelligent transportation systems in proactive safety defense.

[0039] This invention provides a deep learning-based intelligent analysis method for traffic video surveillance images. It offers the following advantages:

[0040] 1. This invention constructs a dual-stream heterogeneous graph network that represents the interaction between dynamic entities and static rules. When the structural tension generated by the vehicle's real trajectory meets the preset triggering conditions, virtual obstacles are instantiated to generate personalized counterfactual trajectories. The correlation between the vehicle's deviation behavior and the assumed risk avoidance behavior is quantified based on the similarity between the trajectory and the real trajectory. This improves the accuracy of traffic behavior analysis by distinguishing between substantive traffic violations and compliant implicit risk avoidance.

[0041] 2. This invention updates the static rules in the dual-flow heterogeneous graph network based on the driving intention determination results, counts the frequency of events that are judged as compliant implicit risk avoidance at preset geographical locations, and creates or updates empirical risk anchor points when preset update conditions are met. This enables the static rules to integrate implicit risk information in historical traffic flow to achieve dynamic marking and adaptation of static hazards in the road environment that are not directly perceived, thereby improving the evolution capability of the dual-flow heterogeneous graph network.

[0042] 3. This invention generates multi-dimensional contextual information that includes driving style and group pattern, and dynamically adjusts the weight of the cost function according to driving style when generating personalized counterfactual trajectories. This ensures that the generated personalized counterfactual trajectories conform to physical kinematic constraints while taking into account the personalized driving characteristics of specific vehicles. This makes the benchmark trajectory used for comparison closer to the driver's instinctive reaction when facing real obstacles, and improves the reliability of the driving intention determination results. Attached Figure Description

[0043] Figure 1 This is a flowchart illustrating the intelligent analysis method for traffic video surveillance images based on deep learning, according to an embodiment of the present invention.

[0044] Figure 2 This is a schematic diagram of the construction of a two-stream heterogeneous graph network according to an embodiment of the present invention;

[0045] Figure 3 This is a schematic diagram illustrating the virtual node instantiation and personalized counterfactual trajectory generation in an embodiment of the present invention;

[0046] Figure 4 This is a schematic diagram of a multidimensional evidence fusion decision model according to an embodiment of the present invention. Detailed Implementation

[0047] The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0048] See attached document Figure 1 This invention provides a method for intelligent analysis of traffic video surveillance images based on deep learning, which includes the following steps:

[0049] S100. Acquire real-time video streams from traffic monitoring and construct a dynamic perception sub-graph in real time based on the video stream. This sub-graph represents the dynamic spatiotemporal information of traffic participants. At the same time, initialize a static knowledge sub-graph based on road asset GIS data. This sub-graph encodes the static structure and rules of the road and establishes a dynamic evolution mechanism so that it can update itself based on subsequent analysis results.

[0050] S200 models multi-dimensional contextual information in traffic scenarios in parallel. On the one hand, it analyzes historical trajectory data of vehicles on normal road sections to create a personalized driving style profile for each vehicle; on the other hand, it identifies the existence of collective trajectory deviation patterns and forms group behavior characteristics by correlating and analyzing the trajectories of multiple vehicles within a specific spatiotemporal window.

[0051] S300 continuously compares the vehicle's actual trajectory with the rules in the static knowledge subgraph and calculates the structural tension between the two in real time. When the vehicle trajectory deviates abnormally, such as crossing the line or driving against the flow of traffic, and no obvious obstacles are detected, causing the structural tension to exceed a preset threshold, the subsequent counterfactual reasoning process is triggered.

[0052] S400: For the identified abnormal behavior, a virtual obstacle node is instantiated at the corresponding position in the static knowledge subgraph to represent potential environmental hazards that are not directly observed by sensors. Subsequently, on the enhanced knowledge subgraph containing the virtual node, combined with the driving style profile established in step S200, a theoretically most reasonable personalized avoidance trajectory, i.e., a counterfactual trajectory, is planned and generated.

[0053] S500: Calculate the similarity between the vehicle's actual trajectory and the generated personalized counterfactual trajectory. Integrate this similarity score with multidimensional evidence such as group behavior characteristics obtained in step S200 to construct a decision model, ultimately determining whether the driving behavior constitutes a substantive traffic violation or a compliant implicit risk avoidance.

[0054] S600. Perform differentiated processing based on the judgment result of step S500. For substantive violations, report relevant evidence to the enforcement platform; for compliant hidden risk avoidance, if it occurs frequently at a certain location, automatically generate a road potential hazard warning and push it to the maintenance department, and trigger the dynamic evolution of the static knowledge subgraph in step S100, solidifying the hazard location as an empirical risk anchor point.

[0055] To better understand the technical solution of the present invention, the technical details involved in each of the above steps will be described in detail below.

[0056] See attached document Figure 2 In this embodiment of the invention, the representation of traffic scenarios is achieved by constructing a dual-stream heterogeneous graph network, which includes a dynamically changing perception subgraph that changes in real time and a static knowledge subgraph with evolutionary capabilities.

[0057] S101. Real-time, multi-angle video data streams are acquired through cameras deployed along the monitored road sections. To eliminate image distortion caused by camera physical parameters and installation angles, and to provide a unified spatial reference for subsequent analysis, the original video frames need to be preprocessed. This preprocessing mainly includes camera intrinsic parameter calibration and coordinate system transformation. The camera matrix and distortion coefficients are obtained through camera intrinsic parameter calibration, and distortion correction is performed on each frame to correct the image distortion caused by lens optical characteristics. Subsequently, a homography transformation matrix is ​​calculated using the pre-calibrated camera extrinsic parameters. The pixel positions in the distortion-corrected image coordinate system are projected and transformed into a unified bird's-eye view coordinate system, i.e., the world coordinate system, proportional to the real-world dimensions, ensuring that subsequent calculations of physical quantities such as trajectory and velocity have true spatial meaning.

[0058] S102, Dynamically Sensing Subgraph Used for real-time capture of dynamic entities and their relationships in traffic scenes. Within each processing cycle, it is generated and updated in the following ways. :

[0059] A pre-trained deep learning object detection model, such as the YOLO series, is applied to the pre-processed video frames. This model performs a forward pass through the video frames and directly outputs the bounding box positions, confidence scores, and categories of all traffic participants in the scene, such as vehicles and pedestrians.

[0060] The detected target information is input into a multi-target tracking module. This module can be implemented using the DeepSORT algorithm, which combines a Kalman filter for motion state prediction and deep appearance features for identity association. For each successfully tracked vehicle... The system assigns a unique tracking ID to each node and continuously updates its state vector. This state vector constitutes the nodes of the dynamic sensing subgraph. Its properties at time Defined as:

[0061] ;

[0062] in, For dynamically sensing nodes in the subgraph. This is the entity's unique tracking ID. For the category of this entity, This represents the position of the entity in the world coordinate system. This is the instantaneous velocity vector of the entity. This is the instantaneous acceleration vector of the entity. The instantaneous velocity and acceleration are obtained by performing first-order and second-order difference calculations on the position changes between consecutive frames.

[0063] Constructing the edge set of a dynamically perceptive subgraph Any two nodes in the graph and The edge between This characterizes the spatial relationship between them. This relationship can be quantified as the Euclidean distance between the two nodes. And used as the weight or attribute of the edge:

[0064] ;

[0065] in, Let be the Euclidean distance between the two nodes. This represents the position of the entity in the world coordinate system. This refers to the position of another entity in the world coordinate system.

[0066] S103, Static Knowledge Subgraph This graph is used to encode the static physical environment and prior knowledge of traffic rules for roads. The construction and evolution process of this graph is as follows:

[0067] Initialize the graph structure. By parsing high-precision road asset GIS data, map the static elements of the roads to nodes and edges of the graph, forming an initial static knowledge subgraph. The node set of the graph Includes entities such as lane lines, traffic signs, width restriction facilities, and known accident-prone areas; each node Store its geographical location, category, and rule attributes. For example, the attribute of a lane line node can be a solid line or a dashed line, and the attribute of a speed limit sign node can be a specific speed value. (Graph edge set) This indicates the physical adjacency or logical association between these static nodes, such as different lane segment nodes being connected by edges to form a complete lane.

[0068] Establish a dynamic evolution mechanism for static knowledge subgraphs. To enable... This approach allows for learning and accumulating experience from subsequent analysis results, introducing an update mechanism for empirical risk anchors. When subsequent steps in this method occur at a specific geographical location... After frequently and with high confidence, the system determines that a reasonable avoidance action has occurred, and then triggers... The system will be updated. Specifically, the system will... Create a new ERA node or enhance an existing ERA node. This node has a confidence weight. Its updates follow these rules:

[0069] ;

[0070] in, For the updated confidence weights, The confidence weights before the update. For learning rate, This mechanism establishes event confidence levels. It transforms static knowledge subgraphs from being entirely static into dynamic, explicit graph structure elements that solidify implicit risk knowledge derived through reasoning, providing more precise prior information for subsequent analysis.

[0071] To achieve accurate judgment of driving intentions, this invention not only analyzes the instantaneous behavior of vehicles, but also performs quantitative modeling of the driver's individual habits and traffic flow patterns in a specific time and space in parallel, in order to construct rich contextual features.

[0072] S201. To distinguish the different reasonable behaviors that different drivers may exhibit when faced with the same situation, the system performs a certain number of checks on each tracked vehicle. Create a dynamic, personalized driving style profile. The process is as follows:

[0073] The system from the vehicle Normal driving segments are selected from historical trajectories. A normal driving segment is defined as a segment where the vehicle's trajectory strictly follows a static knowledge subgraph over a period of time. The defined traffic rules, and no trajectory deviations caused by avoiding other dynamic targets or by implicit risks determined in subsequent steps of this method, occur. From the trajectory sequences of these road segments... In the process, a set of dynamic features reflecting driving habits are extracted. These features include, but are not limited to, the standard deviation of the vehicle's lateral acceleration. Used to characterize its driving stability; the maximum angular velocity of the vehicle when performing lane change or turning operations. This is used to characterize the urgency of its operation; and the vehicle's average acceleration. , used to characterize its acceleration / deceleration preference.

[0074] The extracted dynamic features are combined into a high-dimensional driving style feature vector. The specific form is as follows:

[0075] ;

[0076] in, For driving style feature vectors, Let the standard deviation of the vehicle's lateral acceleration be . The maximum heading angular velocity, This is the average acceleration.

[0077] To eliminate the influence of different features' dimensions and facilitate subsequent processing, the feature vector was normalized. Then, to obtain a concise and easily interpretable driving style profile... This can be achieved by inputting the normalized feature vectors into an unsupervised clustering model. The specific implementation of this model can be the K-Means algorithm. This involves using the feature vectors of all vehicles... Clustering Each cluster represents a typical driving style, for example, when... At that time, they can correspond to aggressive, moderate, and conservative types, respectively. Vehicles Driving style profile This refers to the label or centroid vector of its cluster. This profile will serve as an important constraint when generating personalized counterfactual trajectories later.

[0078] S202. In order to identify collective driving behaviors caused by local and unknown common factors such as road icing and temporary obstacles, the system performs correlation analysis on traffic flow patterns within a specific time and space range.

[0079] Define a spatiotemporal analysis window. This window consists of a geographic region. For example, a circular area with a radius of 50 meters centered on the current point of interest, and a time period. For example, the past 5 minutes are considered together. The system collects the driving trajectories of all vehicles appearing within this spatiotemporal window, forming a trajectory set. ,in This represents the total number of vehicles in the window.

[0080] A clustering algorithm suitable for sequence data is applied to this set of trajectories to identify the existence of similar, anomalous trajectory patterns. A density-based DBSCAN algorithm can be implemented as a specific implementation. When applying the DBSCAN algorithm, a metric for evaluating two trajectories needs to be defined. and The distance metric is determined by the Dynamic Time Warping (DTW) distance, as it effectively calculates the similarity between two sequences of unequal length that may have scaled or shifted over time. A suitable neighborhood radius is then set. After setting the minimum number of samples (MinPts) parameter, the DBSCAN algorithm can divide the trajectory set into several clusters and noise points.

[0081] Quantify clustering results into group behavior characteristics If one or more trajectory clusters appear in the clustering results, and the average shape of the trajectories within each cluster deviates significantly from the static knowledge subgraph... If the defined standard path is not followed, the system determines that a collective deviation pattern exists. (Group behavior characteristics) It can be defined as a vector containing Boolean values ​​and statistics:

[0082] ;

[0083] in, As a characteristic of group behavior, This is a Boolean value indicating whether there is a collective deviation; it is true if a collective deviation exists. The number of trajectories in the anomaly cluster. This represents the average deviation of the anomalous cluster trajectory from the standard path. This group behavior characteristic will serve as strong corroborating evidence for subsequent intent determination.

[0084] In this embodiment of the invention, the initial screening of vehicle behavior is accomplished by calculating the structural tension between its trajectory and static rules. When this tension reaches a certain level and there is no obvious external cause, the system will initiate a more in-depth reasoning mechanism, rather than directly determining it as a violation.

[0085] S301, During the monitoring process, the system continuously tracks each vehicle being monitored. Real-time trajectory points Mapping to static knowledge subgraph In the space, to assess the compliance of its behavior. Structural tension is defined as the relationship between the vehicle trajectory and... A quantitative measure of inconsistencies between traffic rules encoded in the Chinese system.

[0086] Specifically, when the vehicle At any moment trajectory points Intrusion into a When a prohibited area is defined, for example, as a boundary formed by lane line nodes with solid line attributes, the system calculates an instantaneous tension value. This instantaneous tension value can be quantified as the depth to which the trajectory point intrudes into the prohibited area. For example, if the lane line is modeled as a geometric line segment, the instantaneous tension value is the point... The perpendicular distance to the line segment.

[0087] To measure the cumulative effect over a period of time, the system integrates the instantaneous tension value within a sliding time window to obtain the vehicle's... Total structural tension The calculation formula is as follows:

[0088] ;

[0089] in, The total structural tension of the vehicle, The length of the sliding time window. It is an instantaneous tension function. For the trajectory point, This represents the rule set in the static knowledge subgraph. The function has a value of 0 when a trajectory point follows the rules, and a value representing the distance from that point to the compliance boundary when it intrudes into a prohibited area. This integral calculation essentially calculates the depth-time integral area of ​​the vehicle's intrusion into the prohibited area over a period of time, comprehensively reflecting the severity and persistence of the deviation behavior.

[0090] S302, The system sets a predefined structural tension threshold. This threshold can be set based on historical data statistics, for example, by taking the 95th percentile of the tension value generated by occasional, slight deviations during normal driving, to ensure sensitivity to obvious abnormal behavior. When the calculated total structural tension of the vehicle... Exceeding this threshold At this point, the system enters a condition judgment phase.

[0091] At this stage, the system does not immediately classify the behavior as a violation, but first checks whether there is an explicit cause that can be directly observed by the sensors to explain the deviation. Specifically, this is implemented in the dynamic sensing subgraph. In the middle, inspect the vehicle Within a pre-defined safety zone, are there other dynamic entities such as other vehicles or pedestrians? If other nodes exist within the safety zone, and the positions and movement trends of these nodes can reasonably explain the vehicles... Avoidance behaviors, such as a vehicle braking suddenly in front or a vehicle changing lanes from the side, are categorized as normal traffic interactions and do not trigger counterfactual reasoning. The system will only initiate subsequent counterfactual reasoning processes when both of the following conditions are met: the vehicle... Total structural tension Greater than the preset threshold In the dynamic sensing subgraph In the process, no explicit dynamic obstacle could be found that could directly explain the deviation behavior. The design of this triggering logic allows the system to focus its computational resources on dealing with seemingly unprovoked anomalous behaviors, thereby delving into the underlying, unobserved, and latent causes that may exist behind them.

[0092] See attached document Figure 3 After the counterfactual reasoning process is triggered, this embodiment of the invention explores whether the observed abnormal trajectory is a reasonable avoidance by simulating a hypothetical environmental factor and deducing the reasonable behavior that should be under this assumption.

[0093] S401. To simulate a potential environmental hazard that is not directly observed by sensors, the system will... (The sentence is incomplete and requires more context to translate accurately.) Instantiate one or more hypothetical virtual obstacle nodes at their corresponding spatial locations. The instantiation process is adaptive, with its position and attributes determined by the actual trajectory that triggers the inference. The geometric features are used to determine this.

[0094] Specifically, the system analyzes the actual trajectory. The arc segment deviates from the standard path. By calculating the curvature of each point on this arc segment, the point with the maximum curvature is found. The location of the virtual node It is typically set in the direction of the normal at the point of maximum curvature, inside the trajectory arc. This setting is based on the fundamental assumption that the driver's avoidance behavior will generally circumvent the center of the obstacle.

[0095] The influence range of a virtual node, i.e., the size of the area it is considered an obstacle in path planning, is estimated based on the magnitude and length of its deviation from the trajectory. For example, this influence range can be modeled as a circular region with a radius proportional to the maximum vertical distance of the trajectory deviation.

[0096] In addition, this instantiation process also incorporates a static knowledge subgraph. The existing empirical risk anchor point ERA. If there is a high-confidence-weighted ERA node near the area where the actual trajectory deviates. The system will prioritize the location of virtual nodes. Aligning with the location of the ERA node and using its influence range as the initial value for this instantiation, this mechanism allows the inference process to leverage historical experience, improving the accuracy of simulating frequent hazards in specific locations.

[0097] S402, Instantiating virtual nodes The system then constructs a temporary augmented knowledge subgraph for this reasoning process. On this enhanced graph, the system uses a path planning algorithm to deduce a path from the starting point of the vehicle's deviation behavior. To the end of its path back to compliance The theoretically optimal trajectory, i.e., the personalized counterfactual trajectory. .

[0098] In this embodiment, the path planning algorithm can be specifically implemented as Algorithm A. Algorithm A evaluates the cost of each node. The optimal path is found by calculating the following method:

[0099] ;

[0100] in, The value of a node. For a node on the path, From the starting point to the node The actual cost, For the node The estimated cost to reach the destination.

[0101] To generate personalized trajectories that conform to the habits of specific drivers, this method uses the driving style profile established in step S201. It is incorporated into the design of the cost function. Specifically, the actual cost... Designed to be a weighted sum of multiple cost items:

[0102] ;

[0103] in, The actual cost from the starting point to node n. The weighting coefficient for path length cost. For path length cost, The weighting coefficient for path curvature cost. For path curvature cost, The weighting coefficient for the cost of safe distance. Cost of safe distance.

[0104] Driving style profile Its function is to dynamically adjust these weighting coefficients. For example, if the vehicle If the driver's driving style profile is determined to be aggressive, the system will reduce the weight given to curvature cost. This allows the planned counterfactual trajectory to include sharper turns to match the driver's behavior pattern. Conversely, a conservative approach would increase... This results in a smoother generated trajectory. In this way, personalized counterfactual trajectories are generated. It is not only the optimal path to physically avoid virtual obstacles, but also the most likely reasonable avoidance trajectory that closely matches the behavioral patterns of a specific driver.

[0105] See attached document Figure 4 After generating personalized counterfactual trajectories, this embodiment of the invention enters the final decision-making stage. The core of this stage lies in comprehensively utilizing the simulation results of individual behavior and the contextual information of group behavior to make a robust and interpretable determination of the fundamental intent behind the observed abnormal driving behavior.

[0106] S501. In order to quantify the consistency between the vehicle's actual behavior and theoretically reasonable avoidance behavior, the system needs to calculate the vehicle's actual trajectory. Compared with the personalized counterfactual trajectory generated in step S402 The similarity between them.

[0107] Since the two trajectories typically differ in time length and sampling points, this embodiment employs the Dynamic Time Warping (DTW) algorithm for calculation. The DTW algorithm calculates the cumulative distance between the points of the two trajectory sequences by finding the optimal nonlinear alignment path, effectively handling issues such as local scaling, translation, and acceleration / deceleration on the time axis. The specific calculation process of DTW is well-known in the art and will not be elaborated upon here.

[0108] After obtaining the original DTW distance value between the two trajectories Subsequently, to obtain a similarity score with clear physical meaning and ease of subsequent fusion, it needs to be normalized. This embodiment employs an adaptive normalization strategy to convert the original DTW distance into a trajectory similarity score ranging from 0 to 1. :

[0109] ;

[0110] in, Scoring is given based on trajectory similarity. The original DTW distance between the true trajectory and the personalized counterfactual trajectory. For the actual trajectory, To personalize counterfactual trajectories, This is an adaptive normalization factor. The specific value is set to the path length of the longer of the two trajectories. This normalization method makes the physical meaning of the similarity score more intuitive, that is, it approximately represents the ratio of the average point-to-point distance between the true trajectory and the counterfactual trajectory to the total length of the trajectory, effectively avoiding measurement bias caused by the different lengths of the trajectories.

[0111] S502. The final determination of driving intention does not rely solely on a single trajectory similarity score, but rather on a fusion decision model that organically combines evidence of individual behavior with contextual evidence of group behavior. This embodiment employs a classification model based on logistic regression to achieve this fusion decision.

[0112] The model aims to calculate the posterior probability that an event belongs to the category of compliance risk aversion. ,in This represents all available evidence. The model's input is a fused feature vector. This vector contains key evidence in at least two dimensions: individual behavioral evidence, namely the trajectory similarity score calculated in step S501. Evidence of group behavior, namely, the characteristics of group behavior extracted in step S202. Specifically, the Boolean values ​​can be used. and the number of trajectories in the anomaly cluster Fusion of feature vectors The form is:

[0113] ;

[0114] in, To fuse feature vectors, Scoring is given based on trajectory similarity. This is a Boolean value indicating whether there is a collective deviation. This represents the number of trajectories in the abnormal cluster.

[0115] The logistic regression model uses a sigmoid function to map the linear combination of the fused feature vectors into a probability value:

[0116] ;

[0117] in, To determine the posterior probability of compliant risk avoidance, For the Sigmoid function, For the weight vector, To fuse feature vectors, This is the bias term. Weight vector. This reflects the relative contributions of different pieces of evidence to the final decision.

[0118] The final intent determination is based on this posterior probability and a preset decision threshold. For example, 0.5 or the optimal value determined according to the ROC curve: if If the system determines that the driving behavior is a compliant implicit risk avoidance, then the confidence level of this determination is the calculated posterior probability value. When the trajectory similarity... When the value is very high and there is a clear pattern of group behavior, the weight vector The effect of this will significantly increase the posterior probability, thus leading to a high-confidence judgment. If If, in step S402, no counterfactual trajectory can be generated that can reasonably explain the behavior—for example, if an avoidance path does not exist or is too costly—then it is determined to be a substantial traffic violation.

[0119] The final step of this invention aims to transform the analysis and judgment results into actual management actions, and to use these results to optimize the system itself, thereby constructing a complete closed loop from data analysis to entity management and then to knowledge enhancement. This process executes differentiated downstream tasks based on the judgment conclusion of step S500.

[0120] S601. When a driving behavior is ultimately determined to be a substantial traffic violation in step S502, the system will automatically execute the evidence preservation and reporting process. To ensure the completeness and legal validity of the reported information, the system will generate a structured evidence package. This evidence package is a data set, the specific contents of which include the timestamp of the event, the geographical coordinates accurate to latitude and longitude, and the unique tracking ID of the vehicle involved. and categories Extract raw video clips containing, for example, 5 seconds before and after the violation occurred; resulting in the complete real-world trajectory sequence that triggers the analysis. The trajectory data is stored as a sequence of coordinate points in the world coordinate system; the determination process data includes the calculated maximum structural tension. If a counterfactual trajectory is generated, it will include that personalized counterfactual trajectory. Data, trajectory similarity score and the posterior probability output by the final decision model. This includes clearly defined substantive traffic violation labels and corresponding violation types, such as determining a violation of a prohibitory marking based on the attributes of the violated static rule node. This evidence package is encoded in a standardized data format such as JSON or XML and securely pushed to a non-site traffic enforcement platform or a human review center via a pre-defined application programming interface (API) for subsequent enforcement processing.

[0121] S602. When one or more driving behaviors are determined to be compliant implicit risk avoidance in step S502, the system initiates a process for road maintenance and system self-optimization.

[0122] The system records key metadata for each such event, including the geographical location where the event occurred. The system maintains an event accumulator for each geographic location, including the time, the confidence level of the decision, and the time frame.

[0123] The system continuously monitors these accumulators. When in a specific geographical location... When a compliant, implicit risk avoidance event occurs and meets preset trigger conditions, the system will generate a warning for potential road hazards. This trigger condition is a complex logical condition, such as: within a time window. If the frequency of an event exceeds a threshold within a week. Furthermore, the average confidence level of these events is higher than the threshold. Its mathematical expression is:

[0124] ;

[0125] in, A boolean value indicating whether to generate an alert. The geographical location where the event occurred Event count within the time window DeltaTw The geographical location where the event occurred. For time window, For frequency threshold, The confidence level for a single event. This is the average confidence threshold.

[0126] Once an early warning is triggered, the system will generate a structured information message containing the location of the hazard, the frequency of occurrence, the typical avoidance trajectory pattern, and the inferred type of potential risk, such as suspected potholes or suspected obstruction of vision. This message will be pushed to the road maintenance and management department through a message queue or a dedicated interface to provide them with data-driven and precise maintenance decision support.

[0127] At the same time, triggering this warning will also activate the static knowledge subgraph. The dynamic evolution mechanism. For example, in step S103, the system will [determine the location / location]. Create or enhance an empirical risk anchor node (ERA). This solidifies the newly discovered implicit risk knowledge into the system's prior knowledge base. In future analyses, when a vehicle deviates from its trajectory again near this location, the existence of this ERA node will directly provide strong prior information for the instantiation of the virtual node in step S401, thereby improving the efficiency and accuracy of counterfactual reasoning and achieving system self-learning and performance enhancement.

Claims

1. A method for intelligent analysis of traffic video surveillance images based on deep learning, characterized in that, Includes the following steps: By utilizing traffic video streams and geographic information data, a dual-stream heterogeneous graph network representing the interaction between dynamic entities and static rules is constructed. Based on the dual-stream heterogeneous graph network, multi-dimensional contextual information including driving style and group pattern is generated; The structural tension of the vehicle's actual trajectory in the dual-stream heterogeneous graph network is calculated. When the structural tension meets the preset triggering conditions, a counterfactual reasoning triggering command is generated. In response to the counterfactual reasoning trigger command, virtual obstacles are instantiated in the dual-stream heterogeneous graph network and combined with the multi-dimensional contextual information to generate personalized counterfactual trajectories; Calculate the trajectory similarity between the personalized counterfactual trajectory and the real trajectory, and obtain the driving intention determination result based on the trajectory similarity and the multi-dimensional contextual information; The static rules in the dual-stream heterogeneous graph network are updated based on the driving intention determination result to obtain the evolved dual-stream heterogeneous graph network.

2. The intelligent analysis method for traffic video surveillance images based on deep learning according to claim 1, characterized in that, The construction of a two-stream heterogeneous graph network representing the interaction between dynamic entities and static rules includes: The traffic video stream is processed using a target detection model and tracking algorithm, which maps traffic participants as dynamic entity nodes and constructs edges based on the spatial distance between the dynamic entity nodes to form a dynamic perception subgraph. The geographic information data is parsed, the static elements of roads are mapped to static rule nodes, and edges are constructed based on physical connection relationships to form a static knowledge subgraph; The spatial reference of the dynamic perception subgraph and the static knowledge subgraph is unified by a coordinate transformation matrix to obtain the dual-stream heterogeneous graph network.

3. The intelligent analysis method for traffic video surveillance images based on deep learning according to claim 1, characterized in that, The generated multi-dimensional contextual information includes driving style and group pattern, including: Extract the dynamic features of the real trajectory and perform cluster analysis to determine the driving style category to which the vehicle belongs, and obtain the driving style; The correlation analysis of multiple vehicle trajectories within a preset time and space window is performed to identify abnormal trajectory clusters that deviate from the standard path, thus obtaining the group pattern.

4. The intelligent analysis method for traffic video surveillance images based on deep learning according to claim 1, characterized in that, When the structural tension meets the preset triggering condition, a counterfactual reasoning triggering instruction is generated, including: The cumulative depth of the actual trajectory intruding into the prohibited region defined in the static rule is calculated as the structural tension. Determine whether the structural tension exceeds a preset threshold, and detect whether there are dynamic entities in the dual-stream heterogeneous graph network that can explain the deviation behavior; If the structural tension exceeds the preset threshold and the dynamic entity does not exist, the counterfactual reasoning trigger command is output.

5. The intelligent analysis method for traffic video surveillance images based on deep learning according to claim 1, characterized in that, Instantiating virtual obstacles in the dual-stream heterogeneous graph network includes: Calculate the point of maximum curvature of the deviation segment in the actual trajectory; The initial coordinates of the virtual obstacle are determined based on the normal direction of the point of maximum curvature and its position inside the trajectory. If there are empirical risk anchors in the dual-stream heterogeneous graph network, the initial coordinates are corrected using the positions of the empirical risk anchors to obtain the target coordinates and generate the virtual obstacle.

6. The intelligent analysis method for traffic video surveillance images based on deep learning according to claim 1, characterized in that, The generation of personalized counterfactual trajectories includes: In the dual-stream heterogeneous graph network that incorporates the virtual obstacles, a path planning algorithm is used to search for regression paths; A cost function comprising path length, curvature, and safety distance is constructed, and the weights of the cost function are dynamically adjusted according to the driving style to obtain the personalized counterfactual trajectory.

7. The intelligent analysis method for traffic video surveillance images based on deep learning according to claim 1, characterized in that, The obtained driving intention determination result includes: The distance between the true trajectory and the personalized counterfactual trajectory is calculated using a dynamic time warping algorithm to obtain the trajectory similarity. Input the trajectory similarity and the multi-dimensional context information into the classification model to calculate the posterior probability of the compliance risk avoidance category; The driving intention determination result is output based on the posterior probability: if the posterior probability is greater than the preset decision threshold, it is determined to be a compliant implicit risk avoidance; otherwise, it is determined to be a substantive traffic violation.

8. The intelligent analysis method for traffic video surveillance images based on deep learning according to claim 1, characterized in that, Updating the static rules in the dual-stream heterogeneous graph network based on the driving intention determination result includes: Statistics show the frequency of events deemed compliant with implicit risk avoidance at preset geographical locations; When the event frequency meets the preset update conditions, an empirical risk anchor point is created or updated in the preset geographical location of the static rule.

9. The intelligent analysis method for traffic video surveillance images based on deep learning according to claim 8, characterized in that, The empirical risk anchor includes a confidence weight attribute, and creating or updating the empirical risk anchor includes iteratively updating the confidence weight attribute using the confidence level of the event.

10. The intelligent analysis method for traffic video surveillance images based on deep learning according to claim 1, characterized in that, After updating the static rules in the dual-stream heterogeneous graph network based on the driving intention determination result, the method further includes: When the driving intention determination result is a compliant avoidance of hidden risks, a potential road hazard warning is generated, which includes the location and type of the hazard, and the potential road hazard warning is sent to the management terminal.

Images