Deep learning-based key vehicle illegal behavior identification and disposal management method and system

Abstract

The application relates to the technical field of traffic management and deep learning, and discloses a method and system for identifying and managing illegal behaviors of key vehicles based on deep learning. Historical trajectory data, vehicle-mounted sensor data and driver behavior video sequences of key vehicles are acquired, three-dimensional convolutional neural networks and graph convolutional neural networks are used to extract spatiotemporal features and generate a behavior node embedding vector set; a Granger causality test algorithm is used to analyze the time series dependence relationship to generate a causal adjacency matrix; a time series causal graph is constructed to explicitly model the causal chain of driving behavior evolution; after real-time behavior features are mapped to graph nodes, a multi-hop reasoning is performed on the graph attention propagation algorithm, the activation probability distribution of each illegal behavior node in the future time window is calculated, a high-probability illegal behavior node is extracted to generate a pre-judgment result set, and the pre-judgment result set is output to a disposal management system. The application realizes active pre-judgment before illegal behavior occurs, and improves the initiative and prevention effect of disposal management.

Description

Technical Field

[0001] This invention relates to the fields of traffic management, computer vision and deep learning technologies, and more specifically, to a method and system for identifying and managing key vehicle violations based on deep learning. Background Technology

[0002] In the fields of urban traffic management and public safety, the identification of violations by key vehicles (including hazardous materials transport vehicles, dump trucks, and construction vehicles) is of great significance for preventing traffic accidents and ensuring public safety. With the acceleration of urbanization and the continuous growth in the number of vehicles, the need for supervision of key vehicles is becoming increasingly urgent, and the timely identification and handling of violations has become a major challenge for traffic management departments.

[0003] Existing technologies for identifying traffic violations by key vehicles primarily rely on computer vision and deep learning methods. Typical solutions include: object detection algorithms based on convolutional neural networks to identify vehicle regions, followed by classification networks to determine the type of violation; temporal modeling methods based on recurrent neural networks to analyze behavioral patterns in short video clips; and visual feature enhancement methods based on attention mechanisms to improve the accuracy of violation identification. These technologies have, to some extent, improved the automation level of violation identification.

[0004] However, existing technologies have significant limitations. First, they primarily analyze single-frame images or short video clips, enabling detection only after violations have occurred—a post-event identification approach that cannot predict impending violations. Second, they lack the ability to model the causal chain of driving behavior evolution. Driving behavior evolution is a continuous dynamic process; certain preceding behaviors (e.g., continuous lane changes, increased speed fluctuations) have causal dependencies on subsequent violations (e.g., forced lane changes, fatigued driving). However, existing technologies cannot explicitly model these temporal causal relationships, making it impossible to infer future violations from currently observed driving behavior. This post-event identification limitation results in reactive measures lagging behind the actual occurrence of violations, reducing the proactiveness and preventative effectiveness of violation management, and ultimately failing to effectively prevent traffic accidents.

[0005] Therefore, there is an urgent need for a technical solution that can proactively predict illegal behavior before it occurs. By modeling the causal chain of driving behavior evolution, it can infer possible illegal behavior in the future from current driving behavior, thereby achieving early warning and resource allocation, and improving the initiative and prevention effect of key vehicle illegal behavior management. Summary of the Invention

[0006] This invention provides a method and system for identifying and managing key vehicle violations based on deep learning, which solves the technical problems in the prior art of being unable to predict violations in advance and lacking the ability to model the causal chain of driving behavior evolution.

[0007] This invention discloses a method for identifying and managing key vehicle violations based on deep learning, comprising the following steps:

[0008] Historical trajectory data, vehicle sensor data, and driver behavior video sequences of key vehicles are acquired. Spatiotemporal features of the video sequences are extracted using a three-dimensional convolutional neural network. Data from different modalities are aligned according to the time dimension and then stitched together to generate a multimodal temporal feature matrix.

[0009] The multimodal temporal feature matrix is ​​encoded using a graph convolutional neural network to generate a set of behavior node embedding vectors;

[0010] The temporal dependencies of the embedding vector set of the behavior nodes are analyzed using the Granger causality test algorithm to generate a causal adjacency matrix;

[0011] A temporal causal graph is constructed based on the causal adjacency matrix and the behavior node embedding vector set. The nodes of the temporal causal graph represent driving behavior states, and the edges represent causal transition probabilities.

[0012] Obtain the real-time behavior characteristics of key vehicles at the current moment, calculate the similarity between the real-time behavior characteristics and each node in the temporal causal graph, and select the node with the highest similarity as the mapping node.

[0013] Using the graph attention propagation algorithm, multi-hop reasoning is performed on the temporal causal graph starting from the mapping node to calculate the activation probability distribution of each illegal behavior node within the future time window;

[0014] Based on the activation probability distribution, nodes of illegal behavior whose probability values ​​exceed a preset threshold are extracted, and a set of illegal behavior prediction results is generated.

[0015] The predicted results of the illegal acts are output to the handling and management system.

[0016] Further, the step of using the Granger causality test algorithm to analyze the temporal dependencies of the behavioral node embedding vector set and generate a causal adjacency matrix includes: for any two time steps in the behavioral node embedding vector set, using the Granger causality test algorithm to determine whether the behavioral node embedding vector of the previous time step is Granger causally related to the behavioral node embedding vector of the next time step; the Granger causality test algorithm constructs a restricted regression model and a complete regression model, calculates the sum of squared residuals of the restricted regression model and the complete regression model, and calculates the F statistic based on the sum of squared residuals; if the p-value corresponding to the F statistic is less than the significance level threshold, then the previous time step is determined to be Granger causally related to the next time step, and the corresponding element in the causal adjacency matrix is ​​set to 1, otherwise it is set to 0; after traversing all time step pairs, the causal adjacency matrix is ​​output, and the non-zero elements of the causal adjacency matrix indicate that there is a causal dependency between the corresponding time steps.

[0017] Further, the step of constructing a temporal causal graph based on the causal adjacency matrix and the behavioral node embedding vector set includes: using each behavioral node embedding vector in the behavioral node embedding vector set as a graph node feature to construct a graph node set; determining a graph edge set based on the causal adjacency matrix, and establishing directed edges between corresponding nodes if an element in the causal adjacency matrix is ​​1; calculating the edge weight of the directed edge as a causal transition probability, wherein the causal transition probability is obtained by normalizing the inner product of the node embedding vectors, and the normalization calculation normalizes the inner product exponent values ​​of all target nodes pointed to by the source node.

[0018] Further, the step of using the graph attention propagation algorithm to perform multi-hop inference on the temporal causal graph starting from the mapping node and calculating the activation probability distribution of each illegal behavior node within a future time window includes: initializing the activation probability of the mapping node to 1 and the activation probability of other nodes to 0; for each hop inference, traversing each node in the graph, and calculating the activation probability of the node in the current hop based on the activation probabilities of each predecessor node in the predecessor node set, the attention weight from the predecessor node to the current node, and the causal transition probability; the attention weight is obtained by normalizing the score value calculated by concatenating the behavior node embedding vectors of the predecessor node and the current node and performing a nonlinear transformation; after inference for a preset number of hops, obtaining the final activation probability of all nodes in the graph; and extracting the activation probability of nodes belonging to the illegal behavior category in the temporal causal graph based on the pre-labeled illegal behavior node tags to generate the activation probability distribution.

[0019] Furthermore, the three-dimensional convolutional neural network includes an input layer, multiple three-dimensional convolutional layers, pooling layers, and fully connected layers. The input layer receives raw frame data from the driver behavior video sequence, and the fully connected layers output spatiotemporal feature vectors. The convolution operation of the three-dimensional convolutional layers performs convolution operations on both the spatial and temporal dimensions of the input features simultaneously. The convolution output features are obtained by multiplying the weights of the three-dimensional convolutional kernel with the corresponding elements of the input features, summing the results, and adding a bias term. The three-dimensional convolutional neural network adopts a supervised learning training mode, using labeled driving behavior category tags as supervision signals, and is trained by minimizing the cross-entropy loss function through the backpropagation algorithm.

[0020] Furthermore, the graph convolutional neural network comprises multiple graph convolutional layers. The input layer of the graph convolutional neural network receives the node feature matrix and adjacency matrix of the initial feature map, and the output layer outputs the encoded node embedding vector set. The adjacency matrix is ​​constructed based on the temporal proximity relationship between time steps. If two time steps are adjacent within a time window or the interval is less than a preset threshold, an edge connection is established between the corresponding nodes. Each graph convolutional layer obtains the node embedding matrix of the next layer by multiplying the adjacency matrix with the node embedding matrix and the learnable weight matrix of the current layer, and then passing it through a nonlinear activation function. The graph convolutional neural network adopts a self-supervised contrastive learning training mode, which is trained by maximizing the similarity of the embeddings of temporally adjacent nodes within the same action sequence and minimizing the similarity of the embeddings of nodes in different action sequences.

[0021] Furthermore, when calculating the activation probability of the node in the current hop, a time-decay factor is introduced to adjust the propagation intensity of the activation probability; the time-decay factor is a positive number less than 1, and the hop number power of the time-decay factor is multiplied by the activation probability calculated based on the set of predecessor nodes to obtain the activation probability of the current hop; the time-decay factor reflects that the more inference hops there are, the larger the predicted time span and the higher the uncertainty, and the activation probability of nodes with long-term illegal behavior is attenuated and adjusted.

[0022] Furthermore, after constructing the temporal causal graph based on the causal adjacency matrix and the behavioral node embedding vector set, and before obtaining the real-time behavioral features of the key vehicle at the current moment, the method further includes: obtaining the road type, weather conditions, and traffic flow of the key vehicle as environmental context information; calculating the adjusted causal transition probability based on the environmental context information, wherein the adjusted causal transition probability is obtained by multiplying the cause-effect transition probability by the weighted exponent value of the environmental context feature vector; and updating the edge weights of the temporal causal graph using the adjusted causal transition probability to generate a context-adaptive temporal causal graph.

[0023] Furthermore, the step of using the Granger causality test algorithm to analyze the temporal dependencies of the behavior node embedding vector set and generate a causal adjacency matrix also includes generating the causal adjacency matrix using a neural network learning method: constructing a causal relationship learning network, wherein the input layer of the causal relationship learning network receives the behavior node embedding vector set, and the output layer outputs the probability values ​​of the causal adjacency matrix; the forward propagation process of the causal relationship learning network includes: a fully connected layer performing a nonlinear transformation on each node embedding vector, an attention layer calculating the causal relationship score between node pairs, and the output layer generating causal adjacency matrix elements through a sigmoid activation function; the causal relationship learning network adopts a supervised learning training mode, using manually labeled causal relationship tags as supervision signals, and is trained by minimizing the binary cross-entropy loss function; the behavior node embedding vector set is input into the trained causal relationship learning network to generate the causal adjacency matrix.

[0024] This invention also discloses a deep learning-based management system for identifying and handling violations by key vehicles, comprising: a data acquisition module for acquiring historical trajectory data, vehicle sensor data, and driver behavior video sequences of key vehicles, as well as real-time behavioral features at the current moment; a feature extraction module for extracting spatiotemporal features of the video sequences using a three-dimensional convolutional neural network to generate a multimodal temporal feature matrix; a graph encoding module for encoding the multimodal temporal feature matrix using a graph convolutional neural network to generate a set of behavior node embedding vectors; a causal analysis module for analyzing the temporal dependencies of the behavior node embedding vector set using a Granger causality test algorithm to generate a causal adjacency matrix; and a graph structure module. The system comprises the following modules: a construction module for constructing a temporal causal graph based on the causal adjacency matrix and the behavioral node embedding vector set; a node mapping module for calculating the similarity between the real-time behavioral features and each node in the temporal causal graph, and selecting the node with the highest similarity as the mapping node; an inference module for performing multi-hop inference on the temporal causal graph starting from the mapping node using a graph attention propagation algorithm, and calculating the activation probability distribution of each illegal behavior node within a future time window; a prediction module for extracting illegal behavior nodes with probability values ​​exceeding a preset threshold based on the activation probability distribution, and generating a set of illegal behavior prediction results; and an output module for outputting the set of illegal behavior prediction results to the handling and management system.

[0025] This invention addresses the technical problems of existing technologies lacking the ability to model the causal chain of driving behavior evolution and being unable to predict illegal behavior in advance by constructing a temporal causal graph, using the Granger causality test algorithm to identify true causal relationships, and using the graph attention propagation algorithm for multi-hop reasoning. It achieves proactive prediction based on current driving behavior before illegal behavior occurs, resulting in the following technical effects:

[0026] This invention explicitly models the causal chain of driving behavior evolution through a temporal causal graph, overcoming the shortcomings of existing technologies in representing the causal dependencies of behavioral sequences. Nodes in the temporal causal graph represent driving behavior states, and edges represent causal transition probabilities. This graph structure comprehensively depicts the evolutionary path from preceding behaviors to subsequent illegal behaviors, providing a knowledge foundation for predicting illegal behaviors.

[0027] This invention utilizes the Granger causality test algorithm to analyze the temporal dependencies of behavior node embedding vector sets. By comparing the sum of squared residuals of restricted regression and complete regression models, it identifies genuine causal relationships based on statistical significance tests, rather than spurious causal relationships based solely on temporal correlation. This statistically-based causality identification method overcomes the problem of low prediction accuracy caused by existing technologies misclassifying temporal correlation as causal relationships, thus improving the reliability of causal maps.

[0028] This invention utilizes a graph attention propagation algorithm to perform multi-hop inference on a temporal causal graph. By iteratively propagating activation probabilities and dynamically adjusting the contributions of different causal paths using attention weights, it can capture long-term dependencies from the current driving behavior to multiple future time steps. The multi-hop inference process allows the system to extrapolate along the causal chain, even if future violations are temporally distant from the current behavior, their activation probabilities can still be calculated through the causal propagation of intermediate behavior nodes. This overcomes the insufficient temporal perception capability of existing technologies based on single-frame or short-time segment analysis, achieving long-term violation prediction.

[0029] This invention further improves the accuracy and adaptability of the prediction method by introducing a temporal decay factor and adaptive adjustment of environmental context. The temporal decay factor reflects the characteristic that the more inference hops there are, the higher the uncertainty of the prediction, and reasonably decays the long-term prediction; the adaptive adjustment of environmental context enables the temporal causal graph to dynamically adjust the strength of causal relationships according to different driving environments, improving its adaptability to complex and ever-changing traffic scenarios.

[0030] This invention enables early prediction and proactive warning of illegal activities, allowing the management system to trigger warning measures and dispatch law enforcement resources to the predicted location before illegal activities occur. This shifts the focus from post-event handling to pre-event prevention, significantly improving the initiative and preventive effect of key vehicle illegal activities management, helping to reduce traffic accidents and ensure public safety. Attached Figure Description

[0031] Figure 1 This is a schematic diagram of the overall process of the present invention, showing the complete process from acquiring multimodal data to mapping real-time behavioral features to a time-series causal graph;

[0032] Figure 2This is a schematic diagram of the illegal behavior prediction and handling process of the present invention, which shows the process of using graph attention propagation algorithm to perform multi-hop reasoning, generate prediction results and output them to the handling management system;

[0033] Figure 3 This is a flowchart illustrating the sub-steps of the graph attention propagation algorithm of the present invention, showing the detailed steps of activation probability initialization, multi-hop inference calculation, and attention weight calculation;

[0034] Figure 4 This is a schematic diagram of the optimized steps of the present invention, illustrating a method for enhancing the adaptability of temporal causal graphs to different driving scenarios. Detailed Implementation

[0035] In the fields of urban traffic management and public safety, the identification of violations by key vehicles (including hazardous materials transport vehicles, dump trucks, and construction vehicles) is of great significance for preventing traffic accidents and ensuring public safety. Existing technologies for identifying violations by key vehicles mainly rely on the analysis of single-frame images or short video clips, using object detection algorithms to identify vehicle regions and then employing classification networks to determine the type of violation. However, these technologies can only detect violations after they have occurred, falling into the category of post-event identification.

[0036] The technical factor contributing to the aforementioned problems lies in the lack of existing technologies' ability to model the causal chain of driving behavior evolution. The evolution of driving behavior is a continuous and dynamic process, with certain preceding behaviors (such as continuous lane changes and increased speed fluctuations) having a causal dependency on subsequent violations (such as forced lane changes and fatigued driving). Due to the lack of explicit representation and reasoning mechanisms for this temporal causal relationship, existing technologies cannot infer potential future violations from currently observed driving behavior. This results in the inability to predict high-risk violations in advance, causing response measures to lag behind the actual occurrence of violations, thus reducing the initiative and preventative effectiveness of violation management.

[0037] The method described in this embodiment is executed on a computing device equipped with a central processing unit, a graphics processing unit, and a memory, which is connected to a city traffic monitoring system, an on-board sensor network, and a handling management system via a network.

[0038] Step 100: Acquire historical trajectory data, vehicle sensor data, and driver behavior video sequences of key vehicles to generate a multimodal temporal feature matrix;

[0039] By connecting to a traffic monitoring system, GPS trajectory data of key vehicles within a historical time window, speed data, acceleration data, and steering angle data collected by vehicle-mounted sensors, and video sequences of driver behavior recorded by vehicle-mounted cameras are obtained. The GPS trajectory data undergoes time alignment and coordinate normalization, the vehicle-mounted sensor data is filtered and denoised, and the driver behavior video sequences are extracted using a 3D convolutional neural network. The data from these different modalities are then aligned along the time dimension and concatenated to generate a multimodal temporal feature matrix. ,in Indicates the number of time steps. Indicates the feature dimension.

[0040] The aforementioned 3D convolutional neural network comprises an input layer, multiple 3D convolutional layers, pooling layers, and fully connected layers. The input layer receives raw frame data from a video sequence of driver behavior, and the fully connected layers output spatiotemporal feature vectors. The convolution operation of the 3D convolutional layers is defined as follows: ,in Indicates input features, Represents the weights of the 3D convolution kernel. Indicates the bias term. This represents the output feature of the convolution. The aforementioned 3D convolutional neural network employs a supervised learning training mode, using labeled driving behavior category tags as supervisory signals, and minimizing the cross-entropy loss function through backpropagation and Adam optimization algorithms. ,in Indicates the true label, This represents the predicted probability.

[0041] Step 200: Encode the multimodal temporal feature matrix using a graph convolutional neural network to generate a set of behavior node embedding vectors;

[0042] Multimodal temporal feature matrix Features of each time step Considering each node as an initial feature, an initial feature map is constructed. This initial feature map is then encoded using a graph convolutional neural network (Graph Convolutional Neural Network). This Graph Convolutional Neural Network contains multiple graph convolutional layers, and the update rule for each layer is as follows: ,in Indicates the first Layer node embedding matrix, Represents the adjacency matrix. Indicates the first The learnable weight matrix of the layer, This represents a non-linear activation function. After multi-layer graph convolutional encoding, high-dimensional semantic representations corresponding to each time step are obtained, generating a set of behavior node embedding vectors. ,in Indicates the first The embedding vector of behavior nodes at each time step This represents the dimension of the embedding vector.

[0043] The aforementioned graph convolutional neural network's input layer receives the node feature matrix and adjacency matrix of the initial feature map, and the output layer outputs the encoded node embedding vector set. The adjacency matrix is ​​constructed based on the temporal proximity between time steps; if two time steps are adjacent within a time window or the interval is less than a preset threshold, an edge connection is established between the corresponding nodes. The aforementioned graph convolutional neural network employs a self-supervised contrastive learning training mode, maximizing the similarity of temporally adjacent node embeddings within the same action sequence and minimizing the similarity of node embeddings across different action sequences. The loss function is... ,in and For the embedding vector of positive sample pairs, The temperature parameter is used, and the gradient descent optimization algorithm is employed during training.

[0044] Step 300: Analyze the temporal dependencies of the embedding vector sets of behavior nodes using the Granger causality test algorithm, and generate a causal adjacency matrix;

[0045] For any two time steps in the embedding vector set of behavior nodes and ( Using the Granger causality test algorithm to determine the time step Behavior node embedding vector Is it a time step? Behavior node embedding vector Granger causality. The input to the Granger causality test algorithm is the set of behavior node embedding vectors. By constructing restricted regression and complete regression models, the calculations are performed. Statistic ,in and These are the sums of squared residuals for the restricted model and the complete model, respectively. Indicates the difference in the number of parameters. Indicates the number of samples. This indicates the number of parameters in the complete model. If... The statistic corresponds to If the value is less than the significance level threshold, then the time step is determined. For time step Granger causality, in the causal adjacency matrix Settings Otherwise set After traversing all time steps, the Granger causality test algorithm outputs a causal adjacency matrix. The non-zero elements of this matrix indicate that there is a causal dependency between the corresponding time steps.

[0046] Step 400: Construct a temporal causal graph based on the causal adjacency matrix and the behavior node embedding vector set;

[0047] Based on causal adjacency matrix and behavior node embedding vector set Constructing a temporal causal graph Among them, the graph node set Include The node, the The node represents the first... The driving behavior state at each time step is represented by the node feature, which is the corresponding behavior node embedding vector. Graph edge set According to the causal adjacency matrix Confirmed, if Then at node and nodes Establish directed edges between them The edge weights are calculated as causal transition probabilities. ,in Represents the embedding vector of behavior nodes and The inner product of the two driving behavior states reflects the similarity between them. It means that for all satisfying nodes Perform summation. Edge weights. Indicates the state of driving behavior Shift to driving behavior state The probability of causal transition.

[0048] Step 500: Obtain the real-time behavioral characteristics of key vehicles at the current moment and map them to the corresponding nodes in the temporal causal graph;

[0049] Real-time GPS location, speed, acceleration, steering angle, and driver behavior video frames of key vehicles are acquired from traffic monitoring systems and vehicle sensor networks via data connections. The real-time data undergoes the same feature extraction and stitching process as in step 100 to generate a real-time behavior feature vector. Calculate real-time behavior feature vectors. With the behavior node embedding vectors of all nodes in the temporal causal graph Cosine similarity between ,in Representing vectors Norm. Select the node with the highest cosine similarity. As a mapping node of current real-time behavioral characteristics in the temporal causal graph.

[0050] Step 600: Use the graph attention propagation algorithm to perform multi-hop reasoning on the temporal causal graph and calculate the activation probability distribution of each illegal behavior node within the future time window;

[0051] From the mapping node Initially, the graph attention propagation algorithm was used in temporal causal graphs. Up Jump reasoning, among which This represents the preset number of inference hops. The specific execution process of the graph attention propagation algorithm includes the following sub-steps:

[0052] Step 601: Initialize the mapping node The activation probability is 1, that is... The activation probability of other nodes is 0.

[0053] Step 602: For the first Skip reasoning ( ), traverse each node in the graph Calculate its in the first Activation probability of jumping ,in Represents a node The set of predecessor nodes, Indicates attention weights, This represents the probability of causal transition.

[0054] Step 603: Attention Weights The calculation formula is: ,in , This is a learnable attention parameter vector. Represents a node and nodes The behavior node embedding vector concatenation, It is a linear rectified activation function with leakage.

[0055] Step 604: After After skipping inference, obtain the final activation probabilities of all nodes in the graph. .

[0056] Based on the pre-labeled illegal behavior node tags, a subset of nodes belonging to the illegal behavior category in the time-series causal graph is extracted, the activation probability of these illegal behavior nodes is obtained, and the activation probability distribution of each illegal behavior node within a future time window is generated.

[0057] Step 700: Extract illegal behavior nodes whose probability values ​​exceed a preset threshold based on the activation probability distribution, and generate an illegal behavior prediction result set;

[0058] Traverse the activation probability distribution of illegal behavior nodes and assign activation probabilities Exceeding the preset threshold illegal behavior nodes Extract and generate a set of prediction results for illegal activities. ,in This represents a subset of nodes labeled as illegal behavior categories in the time-series causal graph. For each node in the illegal behavior prediction result set, its corresponding illegal behavior type, predicted occurrence time, and activation probability value are extracted to form structured prediction result data.

[0059] Step 800: Output the set of illegal behavior prediction results to the handling management system to trigger early warning and resource scheduling;

[0060] The set of predicted violation results is output to the handling management system via a data interface. Based on the violation type, predicted time of occurrence, activation probability value, and vehicle location information in the prediction results, the handling management system triggers corresponding early warning measures, including sending early warning notifications to law enforcement officers, sending warning messages to vehicle drivers, and dispatching law enforcement resources to stand by near the predicted location of the violation.

[0061] It should be noted that the construction of the aforementioned temporal causal graph can be performed using an offline training method. Specifically, in steps 100 to 400, a large amount of historically accumulated key vehicle behavior data is used for offline training to construct a general temporal causal graph as a knowledge base. In practical applications, steps 500 to 800 utilize this pre-constructed temporal causal graph for real-time reasoning and prediction.

[0062] It should be noted that the generation of the aforementioned causal adjacency matrix can also employ a neural network learning approach. Specifically, in step 300, a causal relationship learning network is constructed. The network takes a set of behavior node embedding vectors as input and outputs a causal adjacency matrix. This network comprises multiple fully connected layers and attention layers, and is trained by minimizing the cross-entropy loss between the predicted causal adjacency matrix and the true causal relationship labels. After training, the behavior node embedding vector set is input into the trained causal relationship learning network to directly generate the causal adjacency matrix.

[0063] The aforementioned causal relationship learning network receives a set of behavioral node embedding vectors as its input layer and outputs the probability values ​​of the causal adjacency matrix as its output layer. The network's forward propagation process includes: a fully connected layer performing a nonlinear transformation on each node's embedding vector. The attention layer calculates the causal relationship score between node pairs. The output layer generates causal adjacency matrix elements through the Sigmoid activation function. The aforementioned causal relationship learning network employs a supervised learning training model, utilizing manually labeled causal relationship tags as supervisory signals, and the loss function is binary cross-entropy. ,in Labels indicating true causal relationship. The Adam optimization algorithm is used for training to predict the causal probability.

[0064] In this embodiment of the application, to improve the accuracy of illegal behavior prediction, step 600 further includes introducing a time-series decay factor to adjust the activation probability propagation intensity. Specifically, in calculating the... When calculating the activation probability of a jump, a time decay factor is introduced. This makes the activation probability calculation formula adjusted to ,in This is the attenuation coefficient. The time-series attenuation factor reflects that the more inference hops there are, the larger the time span of the prediction, and the higher the uncertainty. Therefore, the activation probability of long-term illegal behavior nodes is adjusted by attenuation to avoid over-prediction.

[0065] In this embodiment of the application, in order to enhance the adaptability of the temporal causal graph to different driving scenarios, the following steps are included after step 400 and before step 500:

[0066] Step 410: Obtain environmental context information such as the current road type, weather conditions, and traffic flow of the key vehicle.

[0067] Step 420: Calculate the adjusted causal transition probability based on the environmental context information. The calculation formula is as follows: ,in Represents the environmental context feature vector. This represents the learnable context weight parameters. This represents the adjusted causal transition probability.

[0068] Step 430: Update the edge weights of the temporal causal graph using the adjusted causal transition probabilities to generate a context-adaptive temporal causal graph.

[0069] This optimization step enables the temporal causal graph to dynamically adjust the strength of causal relationships according to different driving environments, thereby improving the adaptability of the prediction method to complex and ever-changing traffic scenarios.

[0070] This implementation explicitly models the causal chain of driving behavior evolution by constructing a temporal causal graph, overcoming the shortcomings of existing technologies in representing the causal dependencies of behavioral sequences. Nodes in the temporal causal graph represent driving behavior states, and edges represent causal transition probabilities. This graph structure comprehensively depicts the evolutionary path from prior behavior to subsequent illegal behavior.

[0071] This implementation utilizes the Granger causality test algorithm to analyze the temporal dependencies of the embedded vector set of behavior nodes. By comparing the sum of squared residuals of the restricted regression model and the complete regression model, it identifies genuine causal relationships based on statistical significance tests, rather than spurious causal relationships based solely on temporal correlation. This causal identification method based on statistical tests overcomes the problem of low prediction accuracy caused by existing technologies misclassifying temporal correlation as causal relationships.

[0072] This implementation utilizes a graph attention propagation algorithm to perform multi-hop inference on a temporal causal graph. By iteratively propagating activation probabilities and dynamically adjusting the contributions of different causal paths using attention weights, it can capture long-term dependencies from the current driving behavior to multiple future time steps. The multi-hop inference process allows the system to extrapolate along the causal chain, even if future violations are temporally distant from the current behavior, their activation probabilities can still be calculated through the causal propagation of intermediate behavior nodes. This overcomes the problem of insufficient temporal perception capabilities caused by existing technologies based on single-frame or short-time segment analysis.

[0073] In summary, this implementation overcomes the problem of insufficient predictive ability caused by the lack of causal modeling in existing technologies by synergistically combining causal chain modeling of temporal causal graphs, identification of true causal relationships through Granger causality tests, and multi-hop reasoning through graph attention propagation. It achieves proactive prediction based on current driving behavior before illegal behavior occurs, enabling the handling and management system to trigger early warnings and allocate resources in advance, thereby improving the initiative and preventive effect of key vehicle illegal behavior management.

Claims

1. A deep learning-based key vehicle illegal behavior identification and disposal management method, characterized in that, Includes the following steps: Historical trajectory data, vehicle sensor data, and driver behavior video sequences of key vehicles are acquired. Spatiotemporal features of the video sequences are extracted using a three-dimensional convolutional neural network. Data from different modalities are aligned according to the time dimension and then stitched together to generate a multimodal temporal feature matrix. The multimodal temporal feature matrix is ​​encoded using a graph convolutional neural network to generate a set of behavior node embedding vectors; The temporal dependencies of the embedding vector set of the behavior nodes are analyzed using the Granger causality test algorithm to generate a causal adjacency matrix; A temporal causal graph is constructed based on the causal adjacency matrix and the behavior node embedding vector set. The nodes of the temporal causal graph represent driving behavior states, and the edges represent causal transition probabilities. Obtain the real-time behavior characteristics of key vehicles at the current moment, calculate the similarity between the real-time behavior characteristics and each node in the temporal causal graph, and select the node with the highest similarity as the mapping node. Using the graph attention propagation algorithm, multi-hop reasoning is performed on the temporal causal graph starting from the mapping node to calculate the activation probability distribution of each illegal behavior node within the future time window; Based on the activation probability distribution, nodes of illegal behavior whose probability values ​​exceed a preset threshold are extracted, and a set of illegal behavior prediction results is generated. The predicted results of the illegal acts are output to the handling and management system.

2. The method for identifying and managing key vehicle violations based on deep learning according to claim 1, characterized in that, The step of analyzing the temporal dependencies of the behavior node embedding vector set using the Granger causality test algorithm to generate a causal adjacency matrix includes: For any two time steps in the set of behavior node embedding vectors, the Granger causality test algorithm is used to determine whether the behavior node embedding vector of the previous time step is Granger causally related to the behavior node embedding vector of the next time step. The Granger causality test algorithm constructs a restricted regression model and a complete regression model, calculates the sum of squared residuals of the restricted regression model and the complete regression model, and calculates the F statistic based on the sum of squared residuals. If the p-value corresponding to the F-statistic is less than the significance level threshold, then the previous time step is determined to be Granger causality of the next time step, and the corresponding element in the causal adjacency matrix is ​​set to 1; otherwise, it is set to 0. After traversing all time step pairs, the causal adjacency matrix is ​​output. The non-zero elements of the causal adjacency matrix indicate that there is a causal dependency between the corresponding time steps.

3. The method for identifying and managing key vehicle violations based on deep learning according to claim 1, characterized in that, The step of constructing a temporal causal graph based on the causal adjacency matrix and the behavioral node embedding vector set includes: The embedding vectors of each behavior node in the embedding vector set are used as graph node features to construct a graph node set; The graph edge set is determined based on the causal adjacency matrix. If an element in the causal adjacency matrix is ​​1, a directed edge is established between the corresponding nodes. The edge weights of the directed edges are calculated as causal transition probabilities. These causal transition probabilities are obtained by normalizing the inner product of the node embedding vectors. The normalization calculation normalizes the inner product exponent values ​​of all target nodes pointed to by the source node.

4. The method for identifying and managing key vehicle violations based on deep learning according to claim 1, characterized in that, The graph attention propagation algorithm is used to perform multi-hop inference on the temporal causal graph, starting from the mapping node, to calculate the activation probability distribution of each illegal behavior node within a future time window, including: The activation probability of the mapped node is initialized to 1, and the activation probability of other nodes is 0. For each hop inference, traverse each node in the graph and calculate the activation probability of the node in the current hop based on the activation probability of each predecessor node in the predecessor node set of the node, the attention weight of the predecessor node to the node, and the causal transition probability. The attention weight is obtained by normalizing the score value calculated by concatenating the behavior node embedding vectors of the predecessor node and the current node and performing a nonlinear transformation. After reasoning through a preset number of hops, the final activation probability of all nodes in the graph is obtained; Based on the pre-labeled illegal behavior node tags, the activation probability of nodes belonging to the illegal behavior category in the time-series causal graph is extracted, and the activation probability distribution is generated.

5. The method for identifying and managing key vehicle violations based on deep learning according to claim 1, characterized in that, The three-dimensional convolutional neural network includes an input layer, multiple three-dimensional convolutional layers, pooling layers, and fully connected layers; The input layer receives raw frame data of the driver behavior video sequence, and the fully connected layer outputs a spatiotemporal feature vector. The convolution operation of the three-dimensional convolutional layer performs convolution operations on the spatial and temporal dimensions of the input features simultaneously. The convolution output features are obtained by multiplying the weights of the three-dimensional convolution kernel with the corresponding elements of the input features, summing the results, and adding a bias term. The three-dimensional convolutional neural network adopts a supervised learning training mode, using labeled driving behavior category tags as supervision signals, and is trained by minimizing the cross-entropy loss function through the backpropagation algorithm.

6. The method for identifying and managing key vehicle violations based on deep learning according to claim 1, characterized in that, The graph convolutional neural network contains multiple graph convolutional layers. The input layer of the graph convolutional neural network receives the node feature matrix and adjacency matrix of the initial feature map, and the output layer outputs the encoded node embedding vector set. The adjacency matrix is ​​constructed based on the temporal proximity relationship between time steps. If two time steps are adjacent within a time window or the interval is less than a preset threshold, an edge connection is established between the corresponding nodes. Each graph convolutional layer obtains the node embedding matrix of the next layer by multiplying the adjacency matrix with the node embedding matrix and the learnable weight matrix of the current layer, and then passing it through a non-linear activation function. The graph convolutional neural network adopts a self-supervised contrastive learning training mode, which is trained by maximizing the similarity of the embeddings of temporally adjacent nodes within the same action sequence and minimizing the similarity of the embeddings of nodes in different action sequences.

7. The method for identifying and managing key vehicle violations based on deep learning according to claim 4, characterized in that, When calculating the activation probability of the node in the current hop, a time-decay factor is introduced to adjust the activation probability propagation intensity; The timing decay factor is a positive number less than 1. The hop count power of the timing decay factor is multiplied by the activation probability calculated based on the set of predecessor nodes to obtain the activation probability of the current hop. The time-series decay factor reflects that the more inference jumps there are, the larger the predicted time span, and the higher the uncertainty. It adjusts the activation probability of nodes of illegal behavior in the long term.

8. The method for identifying and managing key vehicle violations based on deep learning according to claim 1, characterized in that, After constructing the temporal causal graph based on the causal adjacency matrix and the behavior node embedding vector set, and before obtaining the real-time behavior features of the key vehicles at the current moment, the method further includes: Obtain the current road type, weather conditions, and traffic flow of key vehicles as environmental context information; The adjusted causal transition probability is calculated based on the environmental context information. The adjusted causal transition probability is obtained by multiplying the cause-effect transition probability by the weighted exponent value of the environmental context feature vector. The edge weights of the temporal causal graph are updated using the adjusted causal transition probabilities to generate a context-adaptive temporal causal graph.

9. The method for identifying and managing key vehicle violations based on deep learning according to claim 1, characterized in that, The step of analyzing the temporal dependencies of the behavior node embedding vector set using the Granger causality test algorithm to generate a causal adjacency matrix also includes generating the causal adjacency matrix using a neural network learning method. A causal relationship learning network is constructed, wherein the input layer of the causal relationship learning network receives the set of embedding vectors of the behavior nodes, and the output layer outputs the probability values ​​of the causal adjacency matrix; The forward propagation process of the causal relationship learning network includes: a fully connected layer performing a nonlinear transformation on the embedding vector of each node, an attention layer calculating the causal relationship score between node pairs, and an output layer generating causal adjacency matrix elements through a sigmoid activation function. The causal relationship learning network adopts a supervised learning training mode, using manually labeled causal relationship tags as supervision signals, and is trained by minimizing the binary cross-entropy loss function; The behavior nodes are embedded into a vector set and input into a trained causal relationship learning network to generate a causal adjacency matrix.

10. A deep learning-based key vehicle violation identification and handling management system, used to execute the deep learning-based key vehicle violation identification and handling management method according to any one of claims 1 to 9, characterized in that, include: The data acquisition module is used to acquire historical trajectory data of key vehicles, on-board sensor data and driver behavior video sequences, as well as real-time behavioral characteristics at the current moment; The feature extraction module is used to extract the spatiotemporal features of video sequences using a three-dimensional convolutional neural network and generate a multimodal temporal feature matrix. The graph encoding module is used to encode the multimodal temporal feature matrix using a graph convolutional neural network to generate a set of behavior node embedding vectors. The causal analysis module is used to analyze the temporal dependencies of the embedding vector set of the behavior nodes using the Granger causality test algorithm, and generate a causal adjacency matrix. The graph construction module is used to construct a temporal causal graph based on the causal adjacency matrix and the behavior node embedding vector set; The node mapping module is used to calculate the similarity between the real-time behavioral features and each node in the temporal causal graph, and select the node with the highest similarity as the mapping node. The reasoning module is used to perform multi-hop reasoning on the temporal causal graph starting from the mapping node using the graph attention propagation algorithm, and to calculate the activation probability distribution of each illegal behavior node within the future time window; The prediction module is used to extract illegal behavior nodes whose probability values ​​exceed a preset threshold based on the activation probability distribution, and generate a set of illegal behavior prediction results. The output module is used to output the set of predicted illegal behaviors to the handling and management system.

Images