Road traffic anomaly detection and related equipment based on holographic perception

Abstract

The invention provides a road traffic anomaly detection method and related equipment based on holographic perception, and the method comprises the steps: carrying out the time-space registration and preprocessing of video streams, point clouds, vehicle motion states, traffic parameters and environment data collected by multi-source sensor equipment of a target road network, and obtaining a multi-source data stream; performing feature acquisition and fusion processing through a preset target detection model and a feature extraction network to obtain target feature data and scene semantic information; establishing a space-time diagram data structure according to the target feature data and the scene semantic information, and performing target trajectory analysis through a space-time attention mechanism; and carrying out association analysis processing in combination with the analysis result and the environment data to obtain target behavior characteristics and interaction relation data, and realizing road traffic anomaly detection through the target behavior characteristics and the interaction relation data. Through multi-source data fusion and a deep learning model, the accuracy of environment perception is improved, and multi-dimensional information in a complex road network is captured for road traffic anomaly detection.

Description

Technical Field

[0001] The present invention relates to the field of urban management technology, and in particular to a road traffic anomaly detection method based on holographic perception and related equipment. Background Art

[0002] With the rapid development of intelligent transportation and autonomous driving technologies, the demand for perception and dynamic monitoring of road network environments is growing. Traditional traffic perception technologies mainly rely on single sensors or data sources, such as cameras, radars, or vehicle sensors, which cannot fully capture and process multi-dimensional information in complex road networks, resulting in limited perception accuracy and real-time performance. In complex traffic scenarios, the behavior of traffic participants, environmental conditions, and traffic flow are constantly changing. Existing perception systems are difficult to reflect the global status of the road network in real time and comprehensively, especially in terms of multi-source data stream fusion and real-time dynamic updates. Summary of the invention

[0003] The main purpose of the present invention is to solve the technical problem that the existing traffic perception technology cannot fully capture and process the multi-dimensional information in the complex road network, resulting in limited perception accuracy and real-time performance; A first aspect of the present invention provides a road traffic anomaly detection method based on holographic perception, and the road traffic anomaly detection method based on holographic perception comprises: Perform spatiotemporal registration and unified preprocessing on the video stream data, point cloud data, vehicle motion status data, traffic parameter data, and environmental data collected by multi-source sensor devices configured on the target road network to obtain a multi-source data stream in a standard format; The multi-source data stream is subjected to feature acquisition through a preset target detection model and feature extraction network, and feature fusion and semantic segmentation processing are performed using a multi-scale feature fusion network to obtain target feature data of the detected target in the target road network and scene semantic information of the target road network; According to the target feature data and scene semantic information, a spatiotemporal graph data structure is established, and target trajectory analysis is performed based on the spatiotemporal graph data structure through a spatiotemporal attention mechanism to obtain an analysis result; Combining the analysis result with the environmental data, the spatiotemporal graph data structure is subjected to correlation analysis to obtain target behavior characteristics of the detection targets and interaction relationship data between the detection targets; Road traffic anomaly detection is performed on the detection target and the interaction status between the detection targets according to the target behavior characteristics and the interaction relationship data to obtain a road traffic anomaly detection result.

[0004] Optionally, in a first implementation of the first aspect of the present invention, the feature acquisition of the multi-source data stream is performed through a preset target detection model and a feature extraction network, and feature fusion and semantic segmentation processing are performed using a multi-scale feature fusion network to obtain target feature data of the detected target in the target road network and scene semantic information of the target road network, including: Performing target detection and geometric feature extraction processing on the video stream data and point cloud data in the multi-source data stream respectively, and performing motion feature clustering processing on the vehicle motion state data and traffic parameter data to obtain an initial feature set; The features of different scales in the initial feature set are weightedly fused by using preset attention weights, and feature dimension reduction and spatial association processing are performed using a deep convolutional network to obtain a fused feature map; The fused feature map is subjected to multi-level semantic analysis and scene element segmentation processing, and feature matching is performed in combination with a preset semantic mapping relationship library to obtain target feature data of the detected target in the target road network and scene semantic information of the target road network.

[0005] Optionally, in a second implementation of the first aspect of the present invention, a spatiotemporal graph data structure is established according to the target feature data and the scene semantic information, and target trajectory analysis is performed based on the spatiotemporal graph data structure through a spatiotemporal attention mechanism, and the analysis results obtained include: Performing node mapping on a preset spatiotemporal graph data structure template according to the target feature data, and performing edge mapping on the interaction relationship between nodes based on the scene semantic information to obtain an initial spatiotemporal graph structure; Generate a historical trajectory of nodes in the initial spatiotemporal graph structure according to the target feature data and the scene semantic information, and analyze and process the historical trajectory through a spatiotemporal attention algorithm to obtain interaction features between nodes; Using a graph neural network to perform reasoning operations on the initial spatiotemporal graph structure, and combining the interaction features to perform feature fusion to obtain target behavior pattern features; The target behavior pattern feature is matched with a preset behavior pattern library, and abnormal behavior is identified on the target behavior pattern feature in combination with the preset behavior detection rules in the behavior pattern library to obtain the analysis result.

[0006] Optionally, in a third implementation of the first aspect of the present invention, generating a historical trajectory of nodes in the initial spatiotemporal graph structure according to the target feature data and the scene semantic information, and analyzing and processing the historical trajectory through a spatiotemporal attention algorithm to obtain interaction features between nodes includes: Extracting temporal features from target feature data corresponding to nodes in the initial spatiotemporal graph structure, and generating trajectory points in combination with scene constraints in the scene semantic information to obtain an original trajectory sequence of the nodes; Performing trajectory sampling on the original trajectory sequence according to a preset sampling interval, and performing spatial correction on the sampling points in combination with the scene topology structure in the scene semantic information to obtain a standardized trajectory sequence of the node; Dividing the standardized trajectory sequence into time windows, and constructing a trajectory feature matrix by a sliding window method to obtain a historical trajectory representation of the node; The spatiotemporal attention weights between nodes are calculated based on the historical trajectory representation, and the trajectory features are weighted and aggregated according to the attention weights to obtain the interaction features between the nodes.

[0007] Optionally, in a fourth implementation of the first aspect of the present invention, the association analysis processing of the spatiotemporal graph data structure in combination with the analysis result and the environmental data to obtain target behavior characteristics of the detection target and interaction relationship data between the detection targets includes: Performing multi-dimensional feature combination on the abnormal behavior information in the analysis result and the environmental data, and performing feature mapping based on a preset environmental impact model to obtain environmental correlation features; updating the features of the nodes in the spatiotemporal graph data structure according to the environmental association features, and dynamically adjusting the edge weights between the nodes to obtain updated spatiotemporal graph features; Performing graph convolution processing on the updated spatiotemporal graph features, and obtaining a behavior feature vector of the target through multi-layer feature extraction to obtain a target behavior feature of the detected target; Based on the target behavior characteristics, the correlation degree of adjacent nodes in the spatiotemporal graph data structure is calculated, and feature aggregation is performed through a graph attention network to obtain the interaction relationship data between the detection targets.

[0008] Optionally, in a fifth implementation of the first aspect of the present invention, performing road traffic anomaly detection on the detection target and the interaction state between the detection targets according to the target behavior characteristics and the interaction relationship data to obtain a road traffic anomaly detection result includes: Performing state mapping on the dynamic targets in the target road network according to the target behavior characteristics to obtain the behavior state of the dynamic targets; The behavior state is predicted by using the LSTM-GCN hybrid network, and the interactive relationship data is combined for feature supplementation to obtain target state prediction data; Performing multi-scale analysis on the target state prediction data through a preset hierarchical prediction framework to obtain scenario prediction data including individual behavior and traffic flow evolution; Matching the predicted behavior of the dynamic targets in the scenario prediction data with the preset normal behavior rules, and matching the predicted interaction behavior between the dynamic targets with the preset normal interaction rules, to obtain abnormal behaviors of the targets and interactions; Based on the abnormal behavior, abnormal level classification and spatiotemporal correlation analysis are performed on the detection targets in the target road network and the interaction states between the detection targets to obtain the road traffic anomaly detection result.

[0009] Optionally, in a sixth implementation of the first aspect of the present invention, the multi-scale analysis of the target state prediction data through a preset hierarchical prediction framework to obtain scenario prediction data including individual behavior and traffic flow evolution includes: The target state prediction data is sampled in layers according to preset spatiotemporal scales, and feature decomposition of target state prediction data of different scales is performed through a multi-layer feature extraction network to obtain a multi-scale feature sequence; Interactively mapping features at different levels in the multi-scale feature sequence through a preset inter-layer association model to obtain hierarchical association features; The hierarchical correlation features are subjected to spatiotemporal aggregation processing through a preset feature fusion network, and feature reorganization is performed in combination with preset evolution rules to obtain scenario prediction data including individual behavior and traffic flow evolution.

[0010] A second aspect of the present invention provides a road traffic anomaly detection device based on holographic perception, the road traffic anomaly detection device based on holographic perception comprising: The data processing module is used to perform spatiotemporal registration and unified preprocessing on the video stream data, point cloud data, vehicle motion state data, traffic parameter data and environmental data collected by the multi-source sensor devices configured in the target road network, and obtain a multi-source data stream in a standard format; A feature processing module is used to acquire features from multi-source data streams through a preset target detection model and feature extraction network, and to perform feature fusion and semantic segmentation processing using a multi-scale feature fusion network to obtain target feature data of the detected targets in the target road network and scene semantic information of the target road network; An analysis module is used to establish a spatiotemporal graph data structure according to the target feature data and scene semantic information, and perform target trajectory analysis based on the spatiotemporal graph data structure through a spatiotemporal attention mechanism to obtain an analysis result; A behavior interaction analysis module, used to perform association analysis on the spatiotemporal graph data structure in combination with the analysis result and the environmental data, to obtain target behavior characteristics of the detection targets and interaction relationship data between the detection targets; The anomaly detection module is used to perform road traffic anomaly detection on the detection target and the interaction status between the detection targets according to the target behavior characteristics and the interaction relationship data to obtain a road traffic anomaly detection result.

[0011] A third aspect of the present invention provides a computer-readable storage medium, which stores instructions. When the computer-readable storage medium is run on a computer, it enables the computer to execute the steps of the above-mentioned road traffic anomaly detection method based on holographic perception.

[0012] The above-mentioned road traffic anomaly detection and related equipment based on holographic perception obtain multi-source data streams by performing spatiotemporal registration and preprocessing on the video streams, point clouds, vehicle motion states, traffic parameters and environmental data collected by multi-source sensor devices of the target road network; obtain target feature data and scene semantic information by performing feature acquisition and fusion processing through a preset target detection model and feature extraction network; establish a spatiotemporal graph data structure based on the target feature data and scene semantic information, and perform target trajectory analysis through a spatiotemporal attention mechanism; combine the analysis results with environmental data for correlation analysis processing to obtain target behavior characteristics and interactive relationship data, and realize road traffic anomaly detection through target behavior characteristics and interactive relationship data. The present invention improves the accuracy of environmental perception through multi-source data fusion and deep learning models, and captures multi-dimensional information in complex road networks for road traffic anomaly detection.

[0013] Other features and advantages of the present invention will be described in the following description, and partly become apparent from the description, or understood by practicing the present invention. The purpose and other advantages of the present invention are realized and obtained by the structures particularly pointed out in the description, claims and drawings.

[0014] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, preferred embodiments are given below and described in detail with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 Schematic diagram of a first embodiment of a road traffic anomaly detection method based on holographic perception in an embodiment of the present invention; Figure 2 Schematic diagram of an embodiment of a road traffic anomaly detection device based on holographic perception in an embodiment of the present invention. DETAILED DESCRIPTION

[0016] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the accompanying drawings. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0017] The terms "including" and "having" and any variations thereof mentioned in the embodiments of the present invention are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device end including a series of steps or units is not limited to the listed steps or units, but may optionally include other steps or units that are not listed, or may optionally include other steps or units that are inherent to these processes, methods, products or device ends.

[0018] To facilitate understanding of this embodiment, a road traffic anomaly detection method based on holographic perception disclosed in an embodiment of the present invention is first described in detail. Figure 1 As shown, the method comprises the following steps: 101. Performing spatiotemporal registration and unified preprocessing on the video stream data, point cloud data, vehicle motion state data, traffic parameter data, and environmental data collected by the multi-source sensor equipment configured in the target road network to obtain a multi-source data stream in a standard format; In one embodiment of the present invention, the process of performing spatiotemporal registration and unified preprocessing on the data collected by the multi-source sensor devices configured in the target road network first requires the deployment of multi-source sensor devices such as high-definition camera arrays, laser radar sensor networks, millimeter-wave radar arrays, roadside microwave detectors, and environmental sensors at key locations of the target road network. These devices will continuously collect various types of data in the road network, among which the high-definition camera collects video stream information in real time, the laser radar scans to obtain three-dimensional point cloud data, the millimeter-wave radar detects the real-time motion status of the vehicle, the roadside microwave detector records traffic parameters such as traffic volume and speed, and the environmental sensor collects meteorological data such as temperature, humidity, and visibility. While collecting data, a unified coordinate system and time reference are established for the raw data collected by each sensor, and all data are synchronized to the same timestamp to ensure the temporal consistency of the data. Then, preprocessing operations are performed on various types of raw data, and noise suppression is performed on the video stream data by combining Gaussian filtering and median filtering, image quality is improved by image enhancement algorithm, and interference caused by camera jitter is eliminated by video stabilization algorithm. The point cloud data is filtered out using a statistical outlier removal algorithm to remove noise points, and the ICP (iterative closest point) algorithm is used to register point clouds collected from different perspectives, and all point cloud data are aligned to a unified three-dimensional spatial coordinate system. The Kalman filter is applied to radar data for signal smoothing and data calibration to eliminate random errors in the radar measurement process. The environmental data is normalized to unify environmental parameters of different dimensions into a standard range. After completing the preprocessing of each data source, a unified spatiotemporal coordinate transformation matrix is ​​established to transform the data collected by different sensors into the same coordinate system. The spatial correspondence between different sensors is calculated through the internal and external parameter matrix obtained by sensor calibration to achieve spatial alignment of data. At the same time, all data streams are time-aligned based on the GPS time synchronization signal to eliminate the sampling delay of different sensors. Finally, the registered data is encapsulated according to the predefined data format, and a unified data structure is established to organize the video stream, point cloud, vehicle motion status, traffic parameters and environmental data into a standard multi-source data stream to achieve unified storage and efficient access of data. Through the above processing flow, the spatiotemporal alignment and format unification of multi-source heterogeneous data are achieved, solving the heterogeneity of different sensor data in time, space and format, and laying the foundation for subsequent feature extraction and fusion analysis. This collaborative preprocessing method of multi-source data ensures the quality and consistency of the data, allowing the system to fully utilize the advantages of various sensors and obtain rich road network perception information.

[0019] 102. Acquire features of multi-source data streams through a preset target detection model and feature extraction network, and perform feature fusion and semantic segmentation processing using a multi-scale feature fusion network to obtain target feature data of detected targets in the target road network and scene semantic information of the target road network; In one embodiment of the present invention, the feature acquisition of multi-source data streams through a preset target detection model and feature extraction network, and the use of a multi-scale feature fusion network to perform feature fusion and semantic segmentation processing to obtain target feature data of the detected targets in the target road network and the scene semantic information of the target road network include: performing target detection and geometric feature extraction processing on the video stream data and point cloud data in the multi-source data stream, respectively, and performing motion feature clustering processing on the vehicle motion state data and traffic parameter data to obtain an initial feature set; weighted fusion of different scale features in the initial feature set through preset attention weights, and using a deep convolutional network to perform feature dimensionality reduction and spatial association processing to obtain a fused feature map; performing multi-level semantic parsing and scene element segmentation processing on the fused feature map, and performing feature matching in combination with a preset semantic mapping relationship library to obtain the target feature data of the detected targets in the target road network and the scene semantic information of the target road network.

[0020] Specifically, when the video stream data is called by the pre-trained improved YOLOv5 model for target detection, the video frame is first adjusted to the preset resolution (such as 640×640 pixels) and normalized preprocessing is performed. The model output contains multi-scale feature maps, which detect large, medium and small targets at three different scales (such as 80×80, 40×40, and 20×20). The category of the target (such as car, truck, pedestrian, etc.), confidence score and bounding box coordinates are output for each detection box. At the same time, the appearance features of the target are extracted from the middle layer of the backbone network, including visual features such as color, texture, and shape. For point cloud data, when processed by the pre-trained PointNet++ model, the point cloud is first sampled and grouped, the farthest point sampling is used to select key points, and then a ball query is performed around each key point to construct a local point set. The model extracts point cloud features in a layered manner. Each layer contains a sampling layer, a grouping layer and a feature extraction layer, and obtains the geometric features of the target layer by layer, including spatial features such as three-dimensional shape descriptors, local curvature, and normal vectors. When applying the improved DBSCAN algorithm to vehicle motion state data, the feature space is constructed according to the position and velocity vector of the vehicle, and the neighborhood radius and the minimum sample number threshold are set to cluster vehicles with similar motion patterns. The kinematic features such as instantaneous speed, acceleration, heading angle, angular velocity, etc. of each vehicle are extracted from the clustering results. The traffic parameter data is spatiotemporally clustered to calculate the statistical features such as traffic density, average speed, occupancy rate in different time windows and spatial regions. All these features are organized into the form of feature vectors to form the initial feature set. The initial feature set is a multidimensional tensor containing four types of information: target detection features (target category, position, size, etc.), geometric features (three-dimensional shape, posture, etc.), motion features (speed, acceleration, etc.) and statistical features (flow, density, etc.).

[0021] Specifically, the fusion process of features of different scales in the initial feature set first calculates the attention weight. Through the multi-head attention mechanism, the input features are projected into the query, key, and value spaces, and each attention head is responsible for capturing the feature association of a specific mode. In the self-attention calculation, the similarity matrix is ​​calculated for the features in the same modality to obtain the correlation score between the features. In the cross-modal attention calculation, the features of different modalities are mapped to the same feature space, and the degree of association between the modalities is calculated. The attention score is normalized by the softmax function to obtain the attention weight. Then, the attention weight is used to perform weighted fusion of features of different scales. The fusion process considers the spatial position relationship and semantic relevance of the features. The fused features are input into a multi-layer deep convolutional network. The network contains multiple convolutional blocks, each of which consists of a convolutional layer, a batch normalization layer, and an activation function. The size of the convolution kernel is set according to the spatial correlation of the features (such as 3×3, 5×5, etc.), and the step size is set considering the feature dimensionality reduction requirements. Through multi-layer convolution and pooling operations, the feature dimension is gradually reduced, while the fine-grained spatial information is retained through jump connections. The final fused feature map is a four-dimensional tensor consisting of batch dimension, channel dimension and two-dimensional spatial dimension. The channel dimension encodes the fused features of multi-source data, and the spatial dimension retains the location information of the features.

[0022] Specifically, when performing semantic parsing, a hierarchical semantic concept tree is first established, from the basic features at the bottom layer (such as edges and corners) to the component features at the middle layer (such as wheels and body), and then to the high-level semantic concepts (such as vehicle type and behavior mode). The preset hierarchical semantic parsing network contains multiple parsing modules, each of which is responsible for feature extraction at a specific level, and information is transmitted between modules through a context association mechanism. The parsing network adopts an encoder-decoder structure, where the encoder extracts features through multi-layer convolution and the decoder restores spatial resolution through deconvolution. The parallel semantic segmentation branch divides the scene into different semantic areas, including static elements such as lane lines, road surfaces, intersections, and crosswalks, as well as dynamic targets such as vehicles and pedestrians. The segmentation results are matched with the preset semantic mapping relationship library, which is a hierarchical knowledge base that stores standard feature templates and corresponding semantic labels. The semantic category of the feature is determined by calculating the similarity between the feature and the template. The target feature data finally output includes the target's identity information (ID, category), spatial information (position, posture), motion information (speed, trajectory) and other attributes. The scene semantic information includes the scene's constituent elements (road structure, facility distribution), traffic status (traffic characteristics, congestion level), etc. This information is organized in a structured form to facilitate subsequent spatiotemporal analysis and behavior understanding.

[0023] 103. According to the target feature data and scene semantic information, a spatiotemporal graph data structure is established, and the target trajectory analysis is performed based on the spatiotemporal graph data structure through the spatiotemporal attention mechanism to obtain the analysis results; In one embodiment of the present invention, a spatiotemporal graph data structure is established according to the target feature data and scene semantic information, and a target trajectory analysis is performed based on the spatiotemporal graph data structure through a spatiotemporal attention mechanism to obtain an analysis result, including: performing node mapping on a preset spatiotemporal graph data structure template according to the target feature data, and performing edge mapping on the interaction relationship between nodes based on the scene semantic information to obtain an initial spatiotemporal graph structure; generating historical trajectories of nodes in the initial spatiotemporal graph structure according to the target feature data and the scene semantic information, and analyzing and processing the historical trajectories through a spatiotemporal attention algorithm to obtain interaction features between nodes; performing inference operations on the initial spatiotemporal graph structure using a graph neural network, and performing feature fusion in combination with the interaction features to obtain target behavior pattern features; matching a preset behavior pattern library according to the target behavior pattern features, and performing abnormal behavior identification on the target behavior pattern features in combination with the preset behavior detection rules in the behavior pattern library to obtain the analysis result.

[0024] Specifically, we first need to define a standard spatiotemporal graph data structure template. This template is a complete data structure definition, which includes two parts: node structure and edge structure. The node structure defines the attribute fields: unique identifier (for target tracking), target type (category coding of motor vehicles, non-motor vehicles, pedestrians, etc.), timestamp (time series information accurate to milliseconds), spatial coordinates (three-dimensional position information based on a unified coordinate system), motion state (including velocity vector, acceleration vector, heading angle and other kinematic parameters), target size (length, width and height three-dimensional information), feature vector (numerical representation of multimodal features such as visual features and geometric features of the target). The edge structure defines: the unique identifier of the edge, the start node identifier, the end node identifier, the edge type (spatial relationship edge, temporal relationship edge, etc.), the weight value (indicating the strength of association), the time attribute (edge ​​generation time and duration), and the interaction type (follow, intersect, avoid, etc.). On this basis, the input target feature data is node mapped. The mapping process first creates a node instance for each detected dynamic target, and fills the attribute information in the target feature data into the corresponding field of the node: the unique identifier of the node is generated by the target ID, the target category information is encoded and stored in the type field, the spatial position and posture information of the target is converted to a unified coordinate system and stored in the spatial coordinate field, the motion parameters such as speed and acceleration are vectorized and stored in the motion state field, and the feature vector of the target is directly mapped to the feature field. For static targets in the scene (such as traffic signs, street lights, etc.), fixed nodes are created and their attribute information is stored. Then, the edge relationship between nodes is constructed based on the scene semantic information. The construction of the edge follows multi-dimensional association rules: first, the spatial distance relationship between nodes is calculated, and the distance threshold is set to screen potential interactive objects; second, the motion correlation is analyzed, the angle and speed difference of the velocity vector are calculated, and the target pairs related in motion are identified; and finally, the scene semantic information is used to impose constraints to determine whether the targets are in the same lane or adjacent lanes, whether they are in the same road section, etc. On this basis, for each pair of nodes with potential interactions, edge instances are created and edge attributes are set: the edge weight is calculated based on the multi-dimensional degree of association, the edge type label is set according to the interaction type, and the edge timestamp information is recorded. The final initial spatiotemporal graph structure is a complete graph data structure, which is stored in a combination of adjacency matrix and feature matrix. The adjacency matrix A[i][j] represents the edge weight between node i and node j, the feature matrix F[i] stores the feature vector of node i, and the edge attribute information is stored in a separate edge attribute table.

[0025] Specifically, it is necessary to generate historical trajectory data for each node. For each node, the state sequence of the target is extracted from the historical record of the target feature data according to its unique identifier, and the length of the extracted time window is determined according to preset parameters (such as the past 10 seconds). The state snapshot at each time point contains: spatial coordinates (x, y, z three-dimensional position), motion parameters (velocity vector vx, vy, vz, acceleration vector ax, ay, az, heading angle θ, angular velocity ω), timestamp (UNIX timestamp accurate to milliseconds), feature vector (target feature at that moment). The extracted state sequence is interpolated and smoothed. The interpolation uses the cubic spline interpolation method to ensure the smoothness of the trajectory, and the smoothing uses the Savitzky-Golay filter to eliminate abnormal jitter. Then, the trajectory is constrained and corrected using scene semantic information, including: ensuring that the trajectory points do not deviate from the road area according to the road boundary information, projecting the trajectory points to the nearest lane center line according to the lane line information, and filtering unreasonable trajectory segments according to traffic rules (such as one-way lanes, no lane change areas, etc.). The corrected trajectory sequence is organized in chronological order to form a temporal feature matrix, where each row of the matrix corresponds to a complete state description at a time point. Next, the spatiotemporal attention algorithm is used to process these trajectory sequences. The algorithm contains two attention calculation branches: spatial dimension and temporal dimension. In the spatial dimension, the spatial coordinates and motion parameters of the trajectory points are input into the spatial attention module. The spatial attention weight is obtained by calculating the similarity between the query vector (current trajectory point) and the key-value pair (surrounding trajectory points). The weight reflects the importance of different spatial positions. In the temporal dimension, a temporal attention module is constructed, and the state features at different time points are used as input to calculate the attention score in the temporal sequence to capture the temporal dependency of the trajectory. The attention scores of the two dimensions are weighted and summed to obtain a comprehensive spatiotemporal attention weight matrix. Based on the weight matrix, the trajectory sequence is weightedly aggregated to generate the interaction features between nodes. The interaction feature is a high-dimensional feature vector that encodes the spatial interaction pattern (such as the front and rear following distance, the lateral spacing change, the relative speed relationship, etc.) and the temporal interaction law (such as the coordinated change of speed, the acceleration response relationship, the trajectory intersection pattern, etc.) between the targets. These features are stored in the form of feature vectors for subsequent behavior analysis.

[0026] Specifically, when the initial spatiotemporal graph structure is input into a multi-layer graph neural network (GNN) for inference operations, each layer of the network contains two main operations: feature propagation and feature update. In the feature propagation stage, for each node in the graph, its first-order neighbor node set (i.e. directly connected nodes) and second-order neighbor node set (nodes connected through an intermediate node) are first determined. Then, the system designs a message passing function to pass the feature information of the neighbor nodes to the central node. The weight and directionality of the edge are considered in the message passing process, and the information of the neighbor nodes is aggregated by weighted summation.

[0027] For nodes , its feature update formula is: in Represents the characteristics of the node, represents the number of layers of the graph neural network, is a learnable weight matrix, AGG is an aggregation function, For Node The neighbor set of is a nonlinear activation function. In the feature update phase, the system merges the aggregated neighbor information with the node's own features to update the node's state representation. At the same time, the edge features also need to be updated. The update formula for edge features is: in: Represents edge Features, is the update function of the edge feature. In the process of feature transmission, the interaction feature obtained in the previous step is used as a constraint to guide the propagation direction and strength of the feature. In the specific implementation, the interaction feature is converted into an attention coefficient to modulate the message transmission. After multiple layers of graph convolution operations, the features of the nodes gradually integrate the global context information to form target behavior pattern features with rich semantics. These features include the independent behavior patterns of the target (such as basic actions such as acceleration, deceleration, constant speed driving, and turning), interactive behavior patterns (such as following, overtaking, merging, giving way, etc.) and intention information (such as lane change intention, parking intention, etc.). The extracted target behavior pattern features are matched and analyzed with the preset behavior pattern library. The behavior pattern library is a hierarchical knowledge base, which contains three levels: basic behavior layer (storing feature templates of basic actions such as acceleration, deceleration, and turning), interactive behavior layer (storing standard patterns of multi-target interactions), and scene-related layer (storing behavior specifications related to specific scenes). Each behavior pattern template contains: behavior type identifier, feature description vector (standard feature representation of the behavior), spatiotemporal constraints (spatiotemporal restrictions on the occurrence of the behavior), and association rules (association with other behaviors). The matching process first calculates the similarity between the feature to be analyzed and the template feature, using a measurement method such as cosine similarity or Mahalanobis distance. For each behavior type, a matching score is calculated, and behaviors with scores exceeding the preset threshold are considered to be possible behaviors. Then, judgments are made based on the rules in the behavior pattern library, which include a normal behavior rule set and an abnormal behavior rule set. Normal behavior rules define standard behavior patterns, such as normal lane change rules: sufficient safety distance before changing lanes, use of turn signals, smooth lane change process, etc. Abnormal behavior rules define behavior patterns that require warnings, such as sudden acceleration rules: acceleration exceeds the threshold, the distance to the vehicle in front is less than the safety distance, etc. Rule judgment comprehensively considers multiple factors: the degree of deviation between the behavior characteristics and the standard template, the scene conditions where the behavior occurs (such as environmental factors such as weather and road conditions), and the status of surrounding targets (such as the movement status of surrounding vehicles). The final analysis result is a structured data set, which includes: the behavior category determination of each target (marked as normal or abnormal), the specific type and confidence of abnormal behavior, the quantitative evaluation of the degree of abnormality (such as risk level), and related contextual information (such as the cause of the abnormality, other targets affected, etc.). These analysis results are output in a standard data format for subsequent early warning decisions and visual presentation.

[0028] Furthermore, the historical trajectory of the node in the initial spatiotemporal graph structure is generated according to the target feature data and the scene semantic information, and the historical trajectory is analyzed and processed by the spatiotemporal attention algorithm to obtain the interaction characteristics between the nodes, including: extracting the time series features of the target feature data corresponding to the nodes in the initial spatiotemporal graph structure, and generating trajectory points in combination with the scene constraints in the scene semantic information to obtain the original trajectory sequence of the node; sampling the trajectory of the original trajectory sequence according to a preset sampling interval, and spatially correcting the sampling points in combination with the scene topology in the scene semantic information to obtain a standardized trajectory sequence of the node; dividing the standardized trajectory sequence into time windows, and constructing a trajectory feature matrix by a sliding window method to obtain the historical trajectory representation of the node; calculating the spatiotemporal attention weights between nodes based on the historical trajectory representation, and weightedly aggregating the trajectory features according to the attention weights to obtain the interaction characteristics between the nodes.

[0029] Specifically, when extracting the target feature data corresponding to each node from the initial spatiotemporal graph structure, a complete temporal feature extraction system needs to be designed. The system first establishes a feature index table, which contains an index structure of three dimensions: target ID, timestamp, and feature type, to support rapid retrieval and positioning of the target's historical feature data. A dedicated temporal feature extractor is established for each target, which consists of three core modules: state extraction module, temporal association module, and feature organization module. The state extraction module is responsible for extracting multi-dimensional state information from the target feature data: the spatial dimension contains three-dimensional position coordinates (x, y, z) accurate to the centimeter level, and each coordinate value is accompanied by a confidence score; the motion dimension contains a complete set of kinematic parameters, including the velocity vector v=[vx, vy, vz] (recording the velocity components in three directions), the acceleration vector a=[ax, ay, az] (recording the acceleration components in three directions), the heading angle θ (recording the target's orientation), and the angular velocity ω (recording the turning rate); the time dimension contains a UNIX timestamp accurate to the millisecond level and sampling frequency information. The temporal association module establishes the association between states at adjacent moments by constructing a state transition matrix. Each element Pij in the matrix represents the probability of transitioning from state i to state j, and the probability value is calculated by the similarity of the state vector. The feature organization module arranges the extracted state sequence in chronological order and establishes a temporal feature index tree. Each node of the tree stores the state information at a moment, and the connection between nodes represents the temporal relationship of the state. Next, the scene constraints in the scene semantic information are applied to the trajectory point generation process. The constraints are divided into three categories: geometric constraints, rule constraints, and semantic constraints. Geometric constraints include road boundary constraints (using road boundary polygons to limit the spatial range of trajectory points), lane line constraints (using lane line equations to calculate the lateral deviation from the trajectory point to the lane centerline), and curvature constraints (ensuring that the curvature of the trajectory meets the vehicle kinematic characteristics); rule constraints include traffic rule restrictions (one-way lanes, no lane change areas, speed limit areas, etc.), priority rules (priority relationships between primary and secondary roads), and timing rules (signal timing schemes); semantic constraints include scene element relationships (positional relationships with static facilities such as traffic signs and street lights), functional area restrictions (behavioral specifications for special functional areas such as parking areas and U-turn areas), and weather influences (behavioral constraints under different weather conditions). The system constructs a constraint optimization model based on these constraints, optimizes and solves the preliminary trajectory point sequence, and generates trajectory points that meet all constraints. The final node original trajectory sequence is a structured time series data set. Each trajectory point in the data set is a multidimensional feature vector, which contains information such as position coordinates, motion parameters, timestamps, and constraint states. The trajectory points maintain temporal consistency through the temporal association matrix and feature index tree to form a complete motion trajectory description.

[0030] Specifically, a complete trajectory sampling and correction system needs to be designed. The system defines a set of core parameters for trajectory sampling: time sampling interval (the basic interval is set to 0.1 seconds, which can be adjusted dynamically according to the target speed), distance sampling interval (the basic interval is set to 1 meter, which can be adjusted dynamically in key areas), minimum number of sampling points (the lower limit calculated according to the trajectory length to ensure the integrity of the trajectory expression), maximum number of sampling points (the upper limit set considering the computing resource limit), key point judgment threshold (including speed change threshold, direction change threshold, acceleration threshold). The sampling process adopts an adaptive sampling strategy to perform encrypted sampling at the key points of the trajectory. The key points include: speed mutation point (speed change rate exceeds the threshold), direction change point (heading angle change rate exceeds the threshold), acceleration and deceleration point (acceleration exceeds the threshold), and interaction point (the location where interaction occurs with other targets). For each key point, the system uses a smaller sampling interval in the interval before and after it for encrypted sampling, and the size of the sampling interval is inversely proportional to the state change rate. The sampled trajectory point sequence needs to be spatially corrected in combination with the scene topology in the scene semantic information. The correction process includes multiple processing modules: lane alignment module (using B-spline curve to fit the lane centerline, calculating the projection of the trajectory point to the centerline, and correcting the position according to the lateral deviation), intersection correction module (building an intersection trajectory template based on the geometry of the intersection and the turning radius, and mapping the actual trajectory point to the standard template), intersection processing module (using the interpolation algorithm in the trajectory intersection area to ensure the smooth transition of the trajectory), multi-lane processing module (processing the trajectory continuity during the lane change process, including the trajectory morphology of the lane change starting point, lane change process and lane change end point). The corrected trajectory points are organized in a unified data format. Each trajectory point contains: corrected three-dimensional position coordinates, relative position based on the lane reference system, corrected kinematic parameters (speed, acceleration, heading angle, etc.), timestamp information, lane ID, and lane type mark. The final standardized trajectory sequence is stored in a time series database, which supports fast retrieval and spatiotemporal correlation analysis of trajectories, ensuring the consistency and comparability of different target trajectories.

[0031] Specifically, time window division requires the design of a time window division system, which includes a window parameter configuration module and a feature extraction engine. The window parameter configuration module defines a complete window division strategy: basic window size (set to 10 seconds to cover the entire interaction process), sliding step size (set to 1 second to ensure smooth transition between windows), overlap rate (the overlap ratio of adjacent windows is set to 90% to ensure the continuity of features), window weight function (using Gaussian function to assign different weights to data at different times in the window), and window validity judgment conditions (including the minimum number of valid points, data integrity requirements, and noise level thresholds). A dedicated local feature extractor is configured for each time window, and the extractor contains three core modules: the spatial feature module is responsible for extracting the spatial morphological features of the trajectory, including the curvature, steering angle, position distribution, and space occupancy of the trajectory segment; the motion feature module is responsible for extracting dynamic features, including velocity distribution, acceleration distribution, motion mode features, and energy change features; and the time series feature module is responsible for extracting state evolution features, including state transition features, periodic features, and trend features. By moving on the trajectory sequence in a sliding window manner, the feature extraction process is executed for each window position, and the extracted features are organized into a feature matrix. Each row of the feature matrix corresponds to a time window, each column corresponds to a feature dimension, and the elements of the matrix are standardized eigenvalues. The processed data forms a historical trajectory representation, which is a three-dimensional tensor structure containing the time dimension (representing different observation windows), the space dimension (representing the spatial distribution characteristics of the trajectory), and the feature dimension (representing various types of extracted eigenvalues). The feature vector of each window contains complete trajectory description information, supporting subsequent spatiotemporal analysis and pattern recognition.

[0032] Specifically, a complete spatiotemporal attention calculation framework is constructed, which includes a feature conversion module, an attention calculation module, and a feature aggregation module. The feature conversion module first preprocesses the historical trajectory representation, including feature normalization (using the z-score normalization method), dimension alignment (ensuring that all feature vectors have the same dimension), and feature projection (mapping the features to a feature space suitable for attention calculation). The attention calculation module contains two parallel attention units: a spatial attention unit and a temporal attention unit. The spatial attention unit adopts a multi-head attention mechanism, taking the features of the current node as the query vector Q, the features of other nodes as the key vector K and the value vector V, and obtains the attention score of the spatial dimension through the scaled dot product attention calculation. The temporal attention unit adopts a hierarchical attention structure, including local temporal attention (focusing on short-term dependencies) and global temporal attention (capturing long-term dependencies), and calculates the attention score of the temporal dimension through the self-attention mechanism. The attention scores of the two dimensions are fused through a feature-level gating mechanism to obtain a comprehensive spatiotemporal attention weight matrix. The trajectory features are weighted and aggregated according to the attention weight matrix, and the aggregation process considers the importance, temporal dependency, and spatial correlation of the features. The final node-to-node interaction feature is a high-dimensional feature vector that fully encodes the interaction modes (including following mode, overtaking mode, merging mode, avoidance mode, etc.) and spatiotemporal correlation relationships (including speed coordination, trajectory similarity, state synchronization, etc.) between targets.

[0033] 104. Combine the analysis results and the environmental data to perform correlation analysis on the spatiotemporal graph data structure to obtain target behavior characteristics of the detection targets and interaction relationship data between the detection targets; In one embodiment of the present invention, the association analysis processing of the spatiotemporal graph data structure in combination with the analysis results and the environmental data to obtain target behavior characteristics of the detection targets and interaction relationship data between the detection targets includes: performing multi-dimensional feature combination on the abnormal behavior information in the analysis results and the environmental data, and performing feature mapping based on a preset environmental impact model to obtain environmental association characteristics; updating the characteristics of the nodes in the spatiotemporal graph data structure according to the environmental association characteristics, and dynamically adjusting the edge weights between the nodes to obtain updated spatiotemporal graph characteristics; performing graph convolution processing on the updated spatiotemporal graph characteristics, and obtaining the target behavior feature vector through multi-layer feature extraction to obtain the target behavior characteristics of the detection target; calculating the correlation degree of adjacent nodes in the spatiotemporal graph data structure based on the target behavior characteristics, and performing feature aggregation through a graph attention network to obtain interaction relationship data between the detection targets.

[0034] Specifically, a multi-dimensional feature combination processor is constructed to parse the abnormal behavior information in the analysis results. The abnormal behavior information includes: abnormal type coding (such as sudden acceleration, sharp turns, illegal lane changes, etc.), abnormality degree score (normalized score between 0-1), spatiotemporal location of abnormal occurrence, ID set of associated targets, etc. At the same time, the environmental data is characterized, and the environmental data includes: weather conditions (temperature, humidity, visibility, precipitation, etc.), road conditions (friction coefficient, water depth, etc.), lighting conditions (light intensity, lighting angle, etc.), and traffic conditions (flow, density, average speed, etc.). The abnormal behavior information and environmental data are multi-dimensionally associated through the feature combiner, and the combination methods include: time dimension association (aligning environmental data to the time when the abnormality occurs), space dimension association (corresponding environmental data to the location where the abnormality occurs), and state dimension association (establishing a mapping relationship between abnormal state and environmental conditions). Then the preset environmental impact model is called, and the model includes: environmental impact assessor (calculating the impact intensity of environmental factors on abnormal behavior), feature mapper (mapping combined features to standard feature space), and association strength calculator (calculating the degree of association between features). The model outputs the environmental correlation feature, which is a multidimensional feature vector that encodes the correlation between abnormal behavior and environmental factors, including information such as the intensity, direction, and duration of environmental impact.

[0035] Specifically, the spatiotemporal graph data structure is updated based on the environmental association features. First, a feature update processor is established, which includes a node feature update module and an edge weight adjustment module. The node feature update module obtains the influencing factors related to each node in the environmental association features, and adjusts the attribute features of the node according to the influencing factors. The adjustment content includes: the state vector of the node (adjusting the state parameters according to the environmental impact), the behavior characteristics (updating the probability distribution of the behavior pattern), and the risk score (recalculating the risk level according to the environmental conditions). For each node, the original feature vector is fused with the environmental impact factor to generate the updated node feature. The edge weight adjustment module dynamically adjusts the edge weight based on the change of node features. The adjustment process includes: correlation calculation (calculating the feature correlation between node pairs), environmental impact assessment (assessing the impact of environmental factors on node interactions), and weight update (calculating new edge weights based on correlation and environmental impact). The updated spatiotemporal graph features include the impact of environmental factors, which is reflected in the adjustment of node features and the change of edge weights.

[0036] Specifically, deep feature extraction is performed on the updated spatiotemporal graph features to construct a graph convolution processor, which includes multiple graph convolution layers and feature extraction modules. Each graph convolution layer performs the following operations: neighbor aggregation (collecting feature information of adjacent nodes), feature transformation (transforming features through a learnable weight matrix), and nonlinear activation (introducing nonlinearity using activation functions such as ReLU). The stacking of graph convolution layers enables each node to fuse a wider range of contextual information layer by layer. The feature extraction module connects the feature extraction unit after each layer, including: spatial feature extractor (extracting structural features in the spatial domain), semantic feature extractor (extracting abstract features at the semantic level), and temporal feature extractor (extracting dynamic change features). The outputs of these feature extractors are integrated through a feature fusion network to form a behavioral feature vector of the target, which is a high-dimensional feature representation that encodes the complete behavioral pattern and state information of the target.

[0037] Specifically, when building an association analysis processor, first calculate the association between nodes based on the target behavior characteristics. The association calculation module contains multiple calculation units: a spatial association calculator (calculating the mutual influence of nodes in the spatial dimension), a behavior similarity calculator (calculating the similarity of behavior patterns), and a state association calculator (calculating the correlation of state variables). The calculated association is used as the input of the graph attention network. The graph attention network includes: an attention calculation layer (calculating the attention coefficient between nodes), a feature propagation layer (propagating node features based on attention weights), and a feature aggregation layer (fusing multi-layer attention results). The features output by the network undergo final aggregation processing to generate interaction relationship data, which is a structured data set that contains a complete relationship description of the interaction type, interaction intensity, and interaction duration between node pairs.

[0038] 105. Perform road traffic anomaly detection on the detection targets and the interaction status between the detection targets according to the target behavior characteristics and the interaction relationship data to obtain the road traffic anomaly detection result.

[0039] In one embodiment of the present invention, the road traffic anomaly detection is performed on the detection target and the interaction state between the detection targets according to the target behavior characteristics and the interaction relationship data to obtain the road traffic anomaly detection result, which includes: state mapping of the dynamic targets in the target road network according to the target behavior characteristics to obtain the behavior state of the dynamic targets; state prediction of the behavior state is performed through an LSTM-GCN hybrid network, and feature supplementation is performed in combination with the interaction relationship data to obtain target state prediction data; multi-scale analysis is performed on the target state prediction data through a preset hierarchical prediction framework to obtain scenario prediction data including individual behavior and traffic flow evolution; the predicted behavior of the dynamic targets in the scenario prediction data is matched with preset normal behavior rules, and the predicted interaction behavior between dynamic targets is matched with preset normal interaction rules to obtain abnormal behavior of the targets and interactions; based on the abnormal behavior, abnormal level classification and spatiotemporal correlation analysis are performed on the detection targets in the target road network and the interaction state between the detection targets to obtain the road traffic anomaly detection result.

[0040] Specifically, when the target behavior features are mapped to a state, a complete state mapping process is first established. The input target behavior features include the target's kinematic parameters such as speed, acceleration, and steering angle. These parameters are processed by the feature parser. The parser first performs standardized preprocessing on the data, normalizes each parameter to a unified numerical range, and then extracts the target's kinematic feature sequence, including the target's position coordinates, speed value, acceleration value, and attitude angle at continuous time points. At the same time, the target's behavior attributes are extracted, including basic behaviors such as acceleration behavior, deceleration behavior, steering behavior, and lane change behavior. The state mapping process uses a predefined mapping rule set to establish mapping relationships for different types of behaviors, such as the acceleration process is mapped to the acceleration state, and the lane change process is mapped to the lane change state. Physical constraints and traffic rule constraints are considered during mapping to ensure the rationality of the state mapping. The final output dynamic target behavior state is a complete state description, including information such as the current behavior type, behavior duration, and behavior location. This state information is used for subsequent state prediction analysis.

[0041] Specifically, the LSTM-GCN hybrid network is used to analyze and predict the behavior state and interaction relationship. The LSTM structure is responsible for processing the state sequence of a single target. It learns the historical state information through the input gate, forget gate, output gate and memory unit to capture the temporal evolution law of the target behavior. The input data contains the behavior state sequence and context information of the target. The long-term dependency is learned through the gating mechanism to predict the future state of the target. The GCN structure processes the interaction relationship between targets and represents the interaction between targets as a graph structure. The nodes in the graph represent each dynamic target and the edges represent the interaction relationship between targets. Each layer of GCN uses the message passing mechanism to fuse the node features with the neighbor information layer by layer to capture the interaction pattern. The feature supplementation module converts the interaction relationship data into a feature vector, which contains the distance relationship, speed relationship, behavior association and other information between targets. These features are combined with the state prediction results through the fusion layer to obtain the target state prediction data containing complete prediction information.

[0042] Specifically, a hierarchical prediction framework can be called for multi-scale analysis. The framework contains three levels of analysis units: micro, meso, and macro. The micro level analyzes the behavior prediction of individual targets, considers the target's motion constraints, driving characteristics, and road conditions, and predicts the target's trajectory and behavior sequence. The meso level analyzes the traffic pattern in the local area, organizes neighboring targets into interactive subgroups, and analyzes vehicle formation characteristics, local flow density characteristics, and traffic stability indicators. The macro level analyzes the traffic situation of the entire road network and predicts the flow distribution, congestion propagation, and traffic capacity of each area of ​​the road network based on the fluid dynamics model. The prediction results at different levels are integrated through a feature fusion network to output complete scenario prediction data including individual behavior predictions and traffic flow evolution.

[0043] Specifically, abnormal behavior judgment is performed based on the scene prediction data. First, the preset normal behavior rule library is loaded. The rule library defines the standard features of various normal behaviors, including normal acceleration rules, normal lane change rules, normal following rules, etc. The predicted target behavior is matched with rules to determine whether there are three urgent behaviors such as sudden acceleration, sudden deceleration, and sharp turns, whether there are illegal behaviors such as speeding and illegal lane changes, and whether there are dangerous behaviors such as abnormal parking and reverse driving. At the same time, the interactive behaviors predicted between targets are matched with rules to determine whether there are interactive anomalies such as too close distance, rear-end collision risk, and abnormal merging. The matching process calculates the degree of deviation between the behavior characteristics and the rule template, and those greater than the threshold are determined as abnormal behaviors. The final output abnormal behavior data contains information such as the type of anomaly, the location of occurrence, and the target involved.

[0044] Specifically, the detected abnormal behaviors are graded and correlated, and first the abnormal grade classification standard is formulated. Abnormal behaviors are graded according to the degree of danger, scope of impact and duration of abnormal behaviors, such as emergency behaviors are general abnormalities, abnormal parking and speeding are moderate abnormalities, and rear-end collision risks and reverse behaviors are serious abnormalities. Spatiotemporal correlation analysis examines the spatiotemporal distribution characteristics of abnormal behaviors, analyzes the occurrence patterns, propagation trends and correlation relationships of abnormal behaviors, and identifies whether there are chain reactions or mass events. Based on the analysis results, each abnormal behavior is marked with abnormal levels and correlation attributes, and a complete description of the abnormal event is generated, including event type, grade assessment, impact assessment and disposal suggestions, to form the final traffic anomaly detection results.

[0045] Furthermore, the target state prediction data is subjected to multi-scale analysis through a preset hierarchical prediction framework to obtain scenario prediction data including individual behavior and traffic flow evolution, including: stratified sampling of the target state prediction data according to preset spatiotemporal scales, and feature decomposition of target state prediction data of different scales through a multi-layer feature extraction network to obtain a multi-scale feature sequence; interactive mapping processing of features of different levels in the multi-scale feature sequence is performed through a preset inter-layer association model to obtain hierarchical association features; spatiotemporal aggregation processing of the hierarchical association features is performed through a preset feature fusion network, and feature reorganization is performed in combination with preset evolution rules to obtain scenario prediction data including individual behavior and traffic flow evolution.

[0046] Specifically, the target state prediction data is divided into time and space scales. The time and space scale division sets three basic levels: micro layer (time granularity 1 second, spatial range 50 meters), meso layer (time granularity 10 seconds, spatial range 500 meters) and macro layer (time granularity 60 seconds, spatial range 2000 meters). Data sampling is performed at each level, and the sampling process includes time dimension sampling and space dimension sampling. The time dimension sampling uses a sliding window method, the window size is set according to the level, and the sampling interval is determined according to the time granularity. The spatial dimension sampling is based on grid division, the grid size varies with the level, and an adaptive sampling strategy is used to encrypt the sampling in the target dense area. The sampled data is input into a multi-layer feature extraction network. The network contains three parallel feature extraction branches, each branch is responsible for the feature extraction of one level. The micro-level feature extraction branch focuses on individual target features, extracting kinematic features (speed, acceleration, steering, etc.), behavioral features (lane change intention, following vehicle features, etc.) and interaction features (relative relationship with surrounding targets). The meso-level feature extraction branch focuses on local area features, extracting flow features (local density, flow, etc.), queue features (vehicle length, spacing distribution, etc.) and regional features (congestion level, traffic efficiency, etc.). The macro-level feature extraction branch focuses on global features, extracting flow state features (flow distribution, density distribution, etc.), road network features (capacity utilization, key road section status, etc.) and system features (overall service level, bottleneck distribution, etc.). The resulting multi-scale feature sequence is a hierarchical feature data structure, with each layer containing a complete feature description at that scale.

[0047] Specifically, the multi-scale feature sequence is processed through a preset inter-layer association model. The association model includes an uplink mapping module and a downlink mapping module to realize bidirectional information transmission between features at different levels. The uplink mapping module maps low-level features to high-level feature space. The mapping process includes feature aggregation (aggregating microscopic features to form mesoscopic features, and aggregating mesoscopic features to form macroscopic features), scale transformation (adjusting the spatiotemporal resolution of features) and semantic enhancement (abstracting low-level specific features into high-level semantic features). The downlink mapping module maps high-level features to low-level feature space. The mapping process includes constraint transfer (transferring high-level global constraints to low-level), detail supplementation (using high-level information to supplement the details of low-level features) and consistency maintenance (ensuring the semantic consistency of features at different levels). A mapping relationship matrix is ​​established between each pair of adjacent levels, and the matrix elements represent the mapping strength between features. The mapping relationship is adjusted through iterative optimization until the best match between inter-layer features is achieved. The resulting hierarchical association features contain complete cross-scale feature association information, reflecting the mutual influence and constraint relationship between different levels.

[0048] Specifically, the hierarchical association features are input into the preset feature fusion network. The fusion network adopts a multi-branch structure, including a time fusion branch, a space fusion branch, and a semantic fusion branch. The time fusion branch processes feature sequences of different time scales, captures long-term and short-term dependencies through the temporal attention mechanism, and fuses evolutionary features of different time granularities. The spatial fusion branch processes features of different spatial scales, integrates local and global information through the spatial attention mechanism, and forms a unified spatial expression. The semantic fusion branch processes features of different semantic levels, and establishes cross-level semantic associations through concept alignment and semantic mapping. The fused features are matched with the preset evolutionary rule library, which contains traffic flow theoretical models (such as LWR models, mobile tide models, etc.), empirical rule sets (summarized typical evolutionary patterns) and constraints (physical constraints, rule constraints, etc.). The features are reorganized according to the matching results to generate the final scene prediction data. The prediction data is a multi-level dataset that includes individual-level behavior predictions (future trajectory, intention evolution, etc.), local-level flow predictions (density changes, queue evolution, etc.), and global-level system predictions (traffic distribution, bottleneck evolution, etc.). These prediction results constitute a complete description of the evolution of the traffic scene.

[0049] In this embodiment, the multi-source data stream is obtained by performing spatiotemporal registration and preprocessing on the video stream, point cloud, vehicle motion state, traffic parameters and environmental data collected by the multi-source sensor equipment of the target road network; the target feature data and scene semantic information are obtained by performing feature acquisition and fusion processing through the preset target detection model and feature extraction network; according to the target feature data and scene semantic information, a spatiotemporal graph data structure is established, and the target trajectory analysis is performed through the spatiotemporal attention mechanism; the analysis results and environmental data are combined for correlation analysis processing to obtain target behavior characteristics and interaction relationship data, and road traffic anomaly detection is realized through the target behavior characteristics and interaction relationship data. The present invention improves the accuracy of environmental perception through multi-source data fusion and deep learning models, and captures multi-dimensional information in complex road networks for road traffic anomaly detection.

[0050] The above describes the road traffic anomaly detection method based on holographic perception in the embodiment of the present invention. The following describes the road traffic anomaly detection device based on holographic perception in the embodiment of the present invention. Figure 2 In one embodiment of the present invention, a device for detecting abnormal road traffic based on holographic perception includes: The data processing module 201 is used to perform spatiotemporal registration and unified preprocessing on the video stream data, point cloud data, vehicle motion state data, traffic parameter data and environmental data collected by the multi-source sensor devices configured in the target road network to obtain a multi-source data stream in a standard format; The feature processing module 202 is used to acquire features from multi-source data streams through a preset target detection model and feature extraction network, and perform feature fusion and semantic segmentation processing using a multi-scale feature fusion network to obtain target feature data of the detected target in the target road network and scene semantic information of the target road network; An analysis module 203 is used to establish a spatiotemporal graph data structure according to the target feature data and scene semantic information, and perform target trajectory analysis based on the spatiotemporal graph data structure through a spatiotemporal attention mechanism to obtain an analysis result; The holographic display module 204 is used to perform correlation analysis processing on the spatiotemporal graph data structure in combination with the analysis result and the environmental data to obtain target behavior characteristics of the detection targets and interaction relationship data between the detection targets; Road traffic anomaly detection is performed on the detection target and the interaction status between the detection targets according to the target behavior characteristics and the interaction relationship data to obtain a road traffic anomaly detection result.

[0051] In an embodiment of the present invention, the road traffic anomaly detection device based on holographic perception runs the above-mentioned road traffic anomaly detection method based on holographic perception. The road traffic anomaly detection device based on holographic perception obtains a multi-source data stream by performing spatiotemporal registration and preprocessing on the video stream, point cloud, vehicle motion state, traffic parameters and environmental data collected by multi-source sensor devices of the target road network; obtains target feature data and scene semantic information by performing feature acquisition and fusion processing through a preset target detection model and feature extraction network; establishes a spatiotemporal graph data structure based on the target feature data and scene semantic information, and performs target trajectory analysis through a spatiotemporal attention mechanism; combines the analysis results with the environmental data for correlation analysis processing to obtain target behavior features and interactive relationship data, and realizes road traffic anomaly detection through the target behavior features and interactive relationship data. The present invention improves the accuracy of environmental perception through multi-source data fusion and deep learning models, captures multi-dimensional information in complex road networks for road traffic anomaly detection.

[0052] above Figure 2 The road traffic anomaly detection device based on holographic perception in an embodiment of the present invention is described in detail from the perspective of modular functional entities.

[0053] The present invention also provides a computer-readable storage medium, which may be a non-volatile computer-readable storage medium or a volatile computer-readable storage medium. Instructions are stored in the computer-readable storage medium. When the instructions are executed on a computer, the computer executes the steps of the road traffic anomaly detection method based on holographic perception.

[0054] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the above-described system, device, or unit can refer to the corresponding process in the aforementioned method embodiment and will not be repeated here.

[0055] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art or the whole or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk and other media that can store program code.

[0056] As described above, the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features thereof may be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A road traffic anomaly detection method based on holographic perception, characterized in that: The road traffic anomaly detection method based on holographic perception includes: Perform spatiotemporal registration and unified preprocessing on the video stream data, point cloud data, vehicle motion status data, traffic parameter data, and environmental data collected by multi-source sensor devices configured on the target road network to obtain a multi-source data stream in a standard format; The multi-source data stream is subjected to feature acquisition through a preset target detection model and feature extraction network, and feature fusion and semantic segmentation processing are performed using a multi-scale feature fusion network to obtain target feature data of the detected target in the target road network and scene semantic information of the target road network; According to the target feature data and scene semantic information, a spatiotemporal graph data structure is established, and target trajectory analysis is performed based on the spatiotemporal graph data structure through a spatiotemporal attention mechanism to obtain an analysis result; Combining the analysis result with the environmental data, the spatiotemporal graph data structure is subjected to correlation analysis to obtain target behavior characteristics of the detection targets and interaction relationship data between the detection targets; Road traffic anomaly detection is performed on the detection target and the interaction status between the detection targets according to the target behavior characteristics and the interaction relationship data to obtain a road traffic anomaly detection result.

2. The road traffic anomaly detection method based on holographic perception according to claim 1 is characterized in that: The method of acquiring features of multi-source data streams by using a preset target detection model and feature extraction network, and performing feature fusion and semantic segmentation processing by using a multi-scale feature fusion network, to obtain target feature data of the detected target in the target road network and scene semantic information of the target road network includes: Performing target detection and geometric feature extraction processing on the video stream data and point cloud data in the multi-source data stream respectively, and performing motion feature clustering processing on the vehicle motion state data and traffic parameter data to obtain an initial feature set; The features of different scales in the initial feature set are weightedly fused by using a preset attention weight, and a deep convolutional network is used to perform feature dimension reduction and spatial association processing to obtain a fused feature map; The fused feature map is subjected to multi-level semantic analysis and scene element segmentation processing, and feature matching is performed in combination with a preset semantic mapping relationship library to obtain target feature data of the detected target in the target road network and scene semantic information of the target road network.

3. The road traffic anomaly detection method based on holographic perception according to claim 1 is characterized in that: The step of establishing a spatiotemporal graph data structure based on the target feature data and scene semantic information, and performing target trajectory analysis based on the spatiotemporal graph data structure through a spatiotemporal attention mechanism, and obtaining analysis results includes: Performing node mapping on a preset spatiotemporal graph data structure template according to the target feature data, and performing edge mapping on the interaction relationship between nodes based on the scene semantic information to obtain an initial spatiotemporal graph structure; Generate a historical trajectory of nodes in the initial spatiotemporal graph structure according to the target feature data and the scene semantic information, and analyze and process the historical trajectory through a spatiotemporal attention algorithm to obtain interaction features between nodes; Using a graph neural network to perform reasoning operations on the initial spatiotemporal graph structure, and combining the interaction features to perform feature fusion to obtain target behavior pattern features; The target behavior pattern feature is matched with a preset behavior pattern library, and abnormal behavior is identified on the target behavior pattern feature in combination with the preset behavior detection rules in the behavior pattern library to obtain the analysis result.

4. The road traffic anomaly detection method based on holographic perception according to claim 3 is characterized in that: The generating of the historical trajectory of the nodes in the initial spatiotemporal graph structure according to the target feature data and the scene semantic information, and analyzing and processing the historical trajectory by the spatiotemporal attention algorithm to obtain the interaction features between the nodes includes: Extracting temporal features from target feature data corresponding to nodes in the initial spatiotemporal graph structure, and generating trajectory points in combination with scene constraints in the scene semantic information to obtain an original trajectory sequence of the nodes; Performing trajectory sampling on the original trajectory sequence according to a preset sampling interval, and performing spatial correction on the sampling points in combination with the scene topology structure in the scene semantic information to obtain a standardized trajectory sequence of the node; Divide the standardized trajectory sequence into time windows, and construct a trajectory feature matrix by a sliding window method to obtain a historical trajectory representation of the node; The spatiotemporal attention weights between nodes are calculated based on the historical trajectory representation, and the trajectory features are weighted and aggregated according to the attention weights to obtain the interaction features between the nodes.

5. The road traffic anomaly detection method based on holographic perception according to claim 1 is characterized in that: The performing association analysis on the spatiotemporal graph data structure in combination with the analysis result and the environmental data to obtain target behavior characteristics of the detection targets and interactive relationship data between the detection targets includes: Performing multi-dimensional feature combination on the abnormal behavior information in the analysis result and the environmental data, and performing feature mapping based on a preset environmental impact model to obtain environmental correlation features; updating the features of the nodes in the spatiotemporal graph data structure according to the environmental association features, and dynamically adjusting the edge weights between the nodes to obtain updated spatiotemporal graph features; Performing graph convolution processing on the updated spatiotemporal graph features, and obtaining a behavior feature vector of the target through multi-layer feature extraction to obtain a target behavior feature of the detected target; Based on the target behavior characteristics, the correlation degree of adjacent nodes in the spatiotemporal graph data structure is calculated, and the features are aggregated through the graph attention network to obtain the interaction relationship data between the detection targets.

6. The road traffic anomaly detection method based on holographic perception according to claim 1 is characterized in that: The performing road traffic anomaly detection on the detection target and the interaction state between the detection targets according to the target behavior characteristics and the interaction relationship data to obtain the road traffic anomaly detection result comprises: Performing state mapping on the dynamic targets in the target road network according to the target behavior characteristics to obtain the behavior state of the dynamic targets; The behavior state is predicted by using the LSTM-GCN hybrid network, and the interactive relationship data is combined for feature supplementation to obtain target state prediction data; Performing multi-scale analysis on the target state prediction data through a preset hierarchical prediction framework to obtain scenario prediction data including individual behavior and traffic flow evolution; Matching the predicted behavior of the dynamic targets in the scenario prediction data with the preset normal behavior rules, and matching the predicted interaction behavior between the dynamic targets with the preset normal interaction rules, to obtain abnormal behaviors of the targets and interactions; Based on the abnormal behavior, abnormal level classification and spatiotemporal correlation analysis are performed on the detection targets in the target road network and the interaction states between the detection targets to obtain the road traffic anomaly detection result.

7. The method for detecting road traffic anomalies based on holographic perception according to claim 6 is characterized in that: The target state prediction data is subjected to multi-scale analysis through a preset hierarchical prediction framework to obtain scenario prediction data including individual behavior and traffic flow evolution, including: The target state prediction data is sampled in layers according to preset spatiotemporal scales, and feature decomposition of target state prediction data of different scales is performed through a multi-layer feature extraction network to obtain a multi-scale feature sequence; Interactively mapping features at different levels in the multi-scale feature sequence through a preset inter-layer association model to obtain hierarchical association features; The hierarchical correlation features are subjected to spatiotemporal aggregation processing through a preset feature fusion network, and feature reorganization is performed in combination with preset evolution rules to obtain scenario prediction data including individual behavior and traffic flow evolution.

8. A road traffic anomaly detection device based on holographic perception, characterized in that: The road traffic anomaly detection device based on holographic perception includes: The data processing module is used to perform spatiotemporal registration and unified preprocessing on the video stream data, point cloud data, vehicle motion state data, traffic parameter data and environmental data collected by the multi-source sensor devices configured in the target road network, and obtain a multi-source data stream in a standard format; A feature processing module is used to acquire features from multi-source data streams through a preset target detection model and feature extraction network, and to perform feature fusion and semantic segmentation processing using a multi-scale feature fusion network to obtain target feature data of the detected targets in the target road network and scene semantic information of the target road network; An analysis module is used to establish a spatiotemporal graph data structure according to the target feature data and scene semantic information, and perform target trajectory analysis based on the spatiotemporal graph data structure through a spatiotemporal attention mechanism to obtain an analysis result; A behavior interaction analysis module, used to perform association analysis on the spatiotemporal graph data structure in combination with the analysis result and the environmental data, to obtain target behavior characteristics of the detection targets and interaction relationship data between the detection targets; The anomaly detection module is used to perform road traffic anomaly detection on the detection target and the interaction status between the detection targets according to the target behavior characteristics and the interaction relationship data to obtain a road traffic anomaly detection result.

9. A computer-readable storage medium having instructions stored thereon, characterized in that: When the instructions are executed by the processor, the steps of the road traffic anomaly detection method based on holographic perception as described in any one of claims 1 to 7 are implemented.

Images