A traffic event digital twin reproduction method based on vehicle-road cloud perception data

Abstract

The application belongs to the technical field of intelligent transportation, and particularly relates to a traffic event digital twin reproduction method based on vehicle-road cloud perception data; comprising: collecting traffic multi-source heterogeneous data and unifying the spatial coordinate system of the traffic multi-source heterogeneous data and performing space-time alignment; eliminating isolated sensor artifacts and environmental clutter noise to obtain pure observation data; fusing the pure observation data of multiple sensors to obtain multi-source observation optimal fusion estimation of a target and processing the same by using a heterogeneous graph attention network to obtain target feature representation; processing the target feature representation by using a double-layer stacked LSTM to obtain a final time sequence embedding vector and processing the same by using a multilayer perception machine to obtain a traffic event type; automatically intercepting key abnormal interval data containing complete evolution logic according to the traffic event type; inputting the key abnormal interval data into a virtual engine to reproduce the whole process of the traffic event; and the application can accurately detect traffic events and realize traffic event reproduction.

Description

Technical Field

[0001] This invention belongs to the field of intelligent transportation technology, specifically relating to a method for digital twin reconstruction of traffic events based on vehicle-road-cloud perception data. Background Technology

[0002] Traffic incident reconstruction is a core technological support for improving the effectiveness of intelligent transportation safety governance and achieving scientific liability determination. With the in-depth implementation of the "vehicle-road-cloud integration" strategy, reconstructing the evolution process of incidents using comprehensive perception data has become a research focus in the field of intelligent transportation. However, the application of existing technologies in complex scenarios still faces significant challenges.

[0003] Traditional traffic incident debriefing primarily relies on roadside video surveillance playback and manual on-site investigation. Video surveillance is limited by fixed viewpoints and physical obstructions, easily creating blind spots at heavily obstructed urban intersections or highway sections, leading to the loss of crucial facts. Furthermore, the sheer volume of video data makes manual annotation and frame-by-frame playback not only inefficient but also prone to inaccuracies due to subjective judgment biases. More importantly, traditional methods are largely based on a single visual dimension, failing to effectively integrate high-frequency dynamic data such as instantaneous speed, acceleration, torque, and precise posture within the vehicle, making it difficult to reconstruct the instantaneous game and causal evolution logic among multiple stakeholders.

[0004] Under the vehicle-road-cloud collaborative perception architecture, V2X (Vehicle-to-Everything) communication technology brings technological increments to event reproduction. Compared to video streams that often reach megabits per second (Mbps) and have complex encoding and decoding, structured data based on the C-V2X protocol has extremely high transmission energy efficiency. Its lightweight characteristics allow perceived information to be aggregated to the cloud in real time with millisecond-level extremely low latency, and the transmission rate far exceeds that of traditional video data backhaul, ensuring the immediacy and synchronization of data. By directly acquiring the dynamic parameters of connected vehicles through the on-board unit (OBU) and combining the global perspective of the roadside unit (RSU) with high-precision map semantics, the system can overcome the line-of-sight limitation and achieve centimeter-level positioning and full-element perception in non-line-of-sight environments. Summary of the Invention

[0005] To address the shortcomings of existing technologies, this invention proposes a method for digital twin reconstruction of traffic events based on vehicle-road-cloud perception data. This method includes:

[0006] S1: Under the vehicle-road-cloud collaborative perception architecture, collect multi-source heterogeneous traffic data and unify the spatial coordinate system of multi-source heterogeneous traffic data;

[0007] S2: Spatiotemporal alignment of multi-source heterogeneous traffic data after unifying the spatial coordinate system;

[0008] S3: Remove isolated sensor artifacts and environmental clutter noise from multi-source heterogeneous traffic data to obtain clean observation data;

[0009] S4: By fusing clean observation data from multiple sensors, the optimal fusion estimate of the target's multi-source observations is obtained;

[0010] S5: A heterogeneous graph attention network is used to process the optimal fusion estimation of multi-source observations of the target to obtain the target feature representation; a two-layer stacked LSTM is used to process the target feature representation to obtain the final temporal embedding vector;

[0011] S6: Input the final temporal embedding vector into a multilayer perceptron for processing to obtain the traffic event type; automatically extract key abnormal interval data containing complete evolution logic based on the traffic event type;

[0012] S7: Input key abnormal interval data into the virtual engine to reproduce the entire process of the traffic incident.

[0013] Preferably, in step S1, the process of collecting multi-source heterogeneous traffic data includes:

[0014] The vehicle unit on board the intelligent connected vehicle acquires the vehicle's motion state, position coordinates and dynamic parameters in real time, and encapsulates them into standardized basic safety messages;

[0015] Using roadside units and their associated visual and radar sensing equipment, the dynamic spatiotemporal trajectories of other traffic participants in the scene are detected and captured, and encapsulated as sensing sharing messages.

[0016] The system collects phase status and countdown data from roadside traffic signal controllers in real time and integrates road topology key points, lane flow semantics, and geometric and physical constraint information provided by commercial high-precision maps.

[0017] Preferably, step S2 specifically includes:

[0018] Calculate the absolute time offset of multi-source heterogeneous traffic data;

[0019] Multiple kinematic models are run in parallel, and the outputs of the multiple kinematic models are weighted and fused according to the absolute time offset to obtain the final calibrated pose state of the target; the kinematic models include a uniform linear kinematic model, a uniformly accelerated linear kinematic model, and a constant rotation kinematic model.

[0020] Furthermore, the uniformly accelerated linear kinematics model can be expressed as follows:

[0021]

[0022] in, and These represent the initial raw data of the target at the moment of perception data generation. coordinates and coordinate, Indicates calibration delay. Indicates the instantaneous velocity of the target. Indicates the heading angle of the target. Indicates the target's acceleration. This indicates the alignment position after kinematic model derivation; This represents the instantaneous velocity of the target after kinematic model derivation;

[0023] The uniform linear kinematic model is represented as follows:

[0024]

[0025] in, and These represent the target at the observation time. coordinates and coordinate, This represents the initial instantaneous value of the kinematic parameters at the moment the sensing data is generated;

[0026] The constant rotation kinematic model is expressed as:

[0027]

[0028] in, Represents the angular velocity of the target. This represents the heading angle of the target after the kinetic model derivation.

[0029] Furthermore, the weighted fusion of the outputs from multiple kinematic models is represented as follows:

[0030]

[0031] in, Indicates the target is The final calibrated pose state after multi-model weighting at each moment. Indicates the first A kinematic model in The state estimation vector at each moment, Indicates the number of motion models. Indicates the first The posterior probability weights of each sub-model at the current time step This represents the time transition matrix of the corresponding model; This represents the absolute time offset between the data timestamp and the system master clock.

[0032] Preferably, step S3 specifically includes:

[0033] Extract the target's instantaneous position, velocity vector, heading angle, and geometric dimensions, and construct a joint feature state vector;

[0034] Calculate the degree of target correlation based on the joint feature state vector;

[0035] The neighborhood radius threshold is dynamically adjusted based on real-time traffic flow density, and adaptive density clustering is performed based on the neighborhood radius threshold and the degree of correlation with the target. Based on the clustering results, isolated sensor artifacts and environmental clutter noise in multi-source heterogeneous traffic data are removed to obtain clean observation data.

[0036] Preferably, step S4 specifically includes:

[0037] The initial measurement noise covariance matrix of the clean observation data is obtained and real-time dilation correction is performed to generate a dynamic covariance matrix;

[0038] Based on the dynamic covariance matrix, an improved dynamic covariance intersection algorithm is used to fuse clean observation data from multiple sensors to obtain the optimal fusion estimate of the target's multi-source observations.

[0039] Furthermore, an improved dynamic covariance intersection algorithm is used to fuse the clean observation data from multiple sensors, as follows:

[0040]

[0041]

[0042] in, This represents the optimal fusion estimate of multi-source observations of the target. For the first The observation state vector of the target by each sensor. Indicates the first Each sensor weighting coefficient Indicates the first The dynamic covariance matrix of data collected by each sensor Indicates the number of effective multi-source sensors. This represents the optimal covariance matrix of the target after weighted fusion of multi-source observations.

[0043] Preferably, in step S5, the process of using a heterogeneous graph attention network to process the optimal fusion estimation of multi-source observations of the target includes:

[0044] Traffic participants, road topology, and traffic light status are abstracted as graph nodes;

[0045] The GAT network is used to process the optimal fusion estimation of multi-source observations corresponding to graph nodes to obtain the attention weights between graph nodes;

[0046] The target feature representation is calculated based on the attention weights between graph nodes, and the calculation formula is as follows:

[0047]

[0048] in, Represents a node In the Layer feature representation, This represents the activation function. Represents the aggregation coefficient of relation type m. Represents a set of heterogeneous relations. Represents a node For nodes Attention weights This represents the weight matrix of relation type m. Indicates the node at the 1st Layer feature representation , Indicates the heterogeneous relationship type In the topology, with nodes The set of connected neighboring nodes.

[0049] Preferably, in step S5, the target feature representation is processed using a double-layer stacked LSTM, as follows:

[0050]

[0051]

[0052]

[0053]

[0054] in, Represents the final temporal embedding vector. and These represent the first LSTM layer in... The hidden state and memory unit state at any given moment. This represents the first layer LSTM cell. This represents the target feature representation input at the current time. and These represent the first layer LSTM units in... The local hidden state and memory unit state at any given moment. This represents the augmented input vector. This represents the LeakyReLU activation function. and They represent dense fusion operators respectively. The learnable weight matrix and bias vector, This represents the feature concatenation operator. and These represent the second LSTM layer in... The intermediate hidden states and memory unit states at any given moment. This indicates the second-layer LSTM unit. and These represent the second LSTM layer in... The hidden state and memory unit state at any given moment. This represents the residual feedback operator.

[0055] The beneficial effects of this invention are as follows:

[0056] This invention can accurately detect and reproduce traffic incidents. In the data acquisition and preprocessing stage, this invention designs a lightweight multi-source data processing and fusion method for massive heterogeneous data under a vehicle-road-cloud integrated architecture. By constructing a coordinate transformation and multi-model kinematic compensation mechanism, the sampling delay and spatial coordinate deviation between different sensors are calibrated, achieving spatiotemporal alignment of heterogeneous observations on a unified decision plane. An improved Euclidean distance is introduced to represent correlation strength, combined with a parameter adaptive density clustering algorithm, to achieve automatic grouping and deduplication of redundant targets. Based on this, an improved dynamic covariance intersection algorithm is used to extract the global optimal estimate under unknown cross-correlation conditions, transforming fragmented sensing information into a high-quality, standardized structured dataset.

[0057] In the event recognition and interval anchoring stages, this invention proposes a semantic mining method for multi-source sensing data. It utilizes spatial heterogeneous graph attention network modeling to decouple the complex dependencies between agent interactions, map topology, and signal rules, extracting deep spatial features. A temporal adaptive state pool with dynamic ID indexes and a two-layer stacked LSTM model are constructed to enhance the system's ability to capture the time-varying characteristics of traffic flow and long-term behavioral intentions. By introducing an anomaly sensitivity judgment mechanism and assisted supervised training, the model can accurately identify long-tailed abnormal events such as collisions or violations from massive amounts of normal traffic flow and automatically lock the key spatiotemporal trigger intervals of the events, providing a basis for subsequent accurate reproduction.

[0058] In the scene generation and dynamic evolution stages, this invention designs a digital twin reproduction mechanism driven by virtual-real mapping. Using standardized perception fusion data packages as input, a full-element 3D scene model is constructed using Unreal Engine. By establishing a dual-clock synchronization scheduling mechanism and a smoothing algorithm based on quaternion spherical linear interpolation, the adaptation problem between discrete sampling data and continuous rendering frames is solved, ensuring that the motion logic of entities such as vehicles and pedestrians in the virtual space remains highly consistent with the physical real world. Simultaneously, it achieves automated mapping from underlying trajectory data to scene logic evolution, significantly reducing the operational threshold of professional software. Attached Figure Description

[0059] Figure 1 This is a structural diagram of the traffic event digital twin reconstruction method based on vehicle-road-cloud perception data in this invention;

[0060] Figure 2 This is a data flow diagram of the data mining model in this invention;

[0061] Figure 3 This is a structural diagram of the traffic incident reproduction method in this invention. Detailed Implementation

[0062] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0063] This invention proposes a digital twin reconstruction method for traffic events based on vehicle-road-cloud perception data to address issues such as insufficient accuracy in traffic event detection and the high difficulty in reconstructing traffic accidents. For example... Figure 1 As shown, the method includes the following:

[0064] S1: Under the vehicle-road-cloud collaborative perception architecture, collect multi-source heterogeneous traffic data and unify the spatial coordinate system of multi-source heterogeneous traffic data.

[0065] V2X data consists of Basic Safety Message (BSM) and Sensing Sharing Message (SSM). These various data types work together seamlessly across time and space dimensions to detect and reproduce traffic incidents. See Table 1 for details.

[0066] Table 1 V2X Data Sheet

[0067]

[0068] BSM data is central to characterizing vehicle dynamics, providing a real-time benchmark for traffic event reconstruction. It achieves multi-source spatiotemporal alignment through timestamps and locks in absolute pose using latitude, longitude, and altitude. Velocity, acceleration, and braking status fields jointly characterize maneuvering behaviors such as sudden braking or acceleration, while heading angle and angular velocity accurately depict lane-changing and turning trajectories. Furthermore, by combining vehicle dimensions and historical trajectory points, the system can construct a dynamic evolution logic with realistic physical boundaries, providing high-fidelity structured support for multi-body interaction analysis in digital twin scenarios.

[0069] SSM (Site Management System) data is the core of digitally depicting the global roadside awareness situation, providing real-time reference for reproducing the dynamic behavior of non-connected targets in traffic scenarios. It achieves logical alignment with vehicle-side data through timestamps and uses target classification labels to identify pedestrians, non-motorized vehicles, and ordinary vehicles. Position, heading, and velocity vector fields jointly represent the target's trajectory, while target size and confidence scores provide physical boundaries for geometric modeling and conflict determination in the virtual scene. Furthermore, combined with target ID tracking sequences, the system can construct a dynamic evolution logic covering the entire domain, compensating for the blind spots of single-vehicle BSM data.

[0070] Under the vehicle-road-cloud collaborative perception architecture, a perception network covering all elements of physical traffic will be constructed to achieve real-time collection of multi-source heterogeneous traffic data. Specifically:

[0071] The vehicle-mounted unit (OBU) of intelligent connected vehicles is used to obtain the vehicle's motion state, position coordinates, and dynamic parameters in real time, and encapsulate them into standardized basic safety messages.

[0072] Simultaneously utilizing roadside RSUs and their associated visual and radar sensing equipment, the system detects and captures the dynamic spatiotemporal trajectories of other traffic participants, such as non-connected vehicles, pedestrians, and non-motorized vehicles, and encapsulates them into shared sensing messages.

[0073] The system accesses the phase status and countdown data of the roadside traffic signal controller in real time, and deeply integrates road topology key points, lane flow semantics and geometric and physical constraint information provided by commercial high-precision maps.

[0074] The aforementioned multi-source heterogeneous data, covering all elements of "people, vehicles, roads, and environment," is aggregated in real time to edge computing nodes (MEC) via C-V2X wireless communication technology for standardized preprocessing, and then securely transmitted to cloud servers using high-speed network protocols. This constructs structured perception data that supports in-depth analysis of subsequent traffic incidents and dynamic mapping of digital twins.

[0075] To unify the spatial coordinate system of heterogeneous traffic data from multiple sources, specifically: since vehicle-mounted OBUs collect WGS-84 latitude and longitude coordinates based on the Global Positioning System (GPS), while roadside sensing devices and digital twin engines typically use local Cartesian coordinate systems, the system needs to establish a unified spatial reference. Using the Geocentric-Earth-Fixed (ECEF) coordinate system as an intermediary, a transformation matrix is ​​used to map target points to a local Northeast-Eastern-Heaven (ENU) station-centered coordinate system with the roadside RSU as the origin. During the transformation process, the radius of curvature of the maxima and minima is first calculated, and then the spatial rectangular coordinates are obtained. This spatial projection transformation eliminates spatial displacement deviations caused by different sensing sources due to different reference systems.

[0076] S2: Spatiotemporal alignment of multi-source heterogeneous traffic data after unifying the spatial coordinate system.

[0077] The system calculates the absolute time offset of multi-source heterogeneous traffic data. Specifically, in a C-V2X communication environment, there is a fluctuating time delay from the acquisition of raw data packets to processing at edge nodes, and the sampling frequencies of heterogeneous sensors are often inconsistent. The system accurately calculates the absolute time offset of each observation in the multi-source heterogeneous traffic data by extracting the UTC timestamp built into the data packet and comparing it with the master clock of the edge node system.

[0078] Multiple kinematic models are run in parallel, and their outputs are weighted and fused based on the absolute time offset to obtain the final calibrated pose state of the target. Specifically:

[0079] The system enters the kinematic state prediction stage, and a dynamic compensation algorithm based on interactive multi-model (IMM) is constructed to address the complex maneuvering characteristics of traffic participants. This algorithm runs multiple kinematic models in parallel, including uniform velocity linear (CV), uniform acceleration linear (CA), and constant tachogeneration (CT), and uses the state transition matrix to perform forward extrapolation of the target's displacement during the offset time.

[0080] The CA uniform acceleration model assumes a calibration delay. The interior target has a constant longitudinal acceleration and no steering. Assume the target's state vector at the observation time includes its position. ,rate Heading angle and acceleration Under the CA model, the formula for the calibrated pose state is as follows:

[0081]

[0082] In the formula, The alignment position after deduction; The calibrated instantaneous rate, and These represent the initial raw data of the target at the moment of perception data generation. coordinates and coordinate.

[0083] The CV uniform linear motion model is based on the assumption of a constant velocity vector in the short term and is suitable for pedestrians or cruising vehicles with simple motion patterns. The calibration state of the system at any given time can be calculated by linear extrapolation as shown in the formula:

[0084]

[0085] in, This indicates the initial instantaneous value of the kinematic parameter within the parentheses at the moment the sensing data is generated.

[0086] The CT constant rotation rate model assumes calibration delay. Internal target angular rate The trajectory derivation is extended from a straight line to an arc mapping by embedding rotational kinematics equations, thus effectively compensating for the lateral offset caused by vehicle steering. Under the CT assumption, the heading angle follows a linear evolution. By integrating the instantaneous velocity vector in the time domain, the calibrated state formula is derived as shown below:

[0087]

[0088] In the formula, The alignment position after deduction. At this moment, it represents the equivalent radius of curvature parameter. Represents the angular velocity of the target. This represents the heading angle of the target after the kinetic model derivation.

[0089] Ultimately, within the IMM prediction framework, the system maintains a set of model probability vectors in real time. , , and These represent the posterior probability weights of the uniform linear motion model, the uniformly accelerated linear motion model, and the constant turning rate motion model at the current moment, respectively; the final calibration state of the target. The weighted fusion formula based on the prediction results of each motion model is as follows:

[0090]

[0091] in, Indicates the target is The final integrated state estimation vector after multi-model weighting at each time step is the final calibrated pose state. Indicates the first The state estimation vector of a kinematic model at time t. Indicates the number of motion models. Indicates the first The posterior probability weights of each sub-model at the current time step This represents the time transition matrix of the corresponding model; This represents the absolute time offset between the data timestamp and the system master clock.

[0092] To obtain the posterior probability weights, the deviation between the predicted values ​​and the actual observed values ​​of each model is first calculated; then, the likelihood function value of each model is calculated based on the assumed Gaussian distribution; finally, the posterior probability is calculated using Bayes' theorem by combining the Markov transition matrix and normalized, and used as the new weights of each model at the current time step.

[0093] The above process enables unified calibration of different targets in time and space. The targets specifically refer to various dynamic entities in traffic scenarios, mainly including motor vehicles, non-motor vehicles, and pedestrians.

[0094] S3: Remove isolated sensor artifacts and environmental clutter noise from multi-source heterogeneous traffic data to obtain clean observation data.

[0095] In a spatiotemporally aligned data sequence, a joint feature state vector is constructed by extracting the target's instantaneous position, velocity vector, heading angle, and geometric dimensions.

[0096] The degree of correlation between targets is calculated based on the joint feature state vector. Specifically, the system uses weighted Euclidean distance as a similarity criterion to calculate the degree of correlation between targets observed by different sensing sources. The measurement formula is as follows: Represented as:

[0097]

[0098] in, This indicates the degree of correlation between target i and target j. The weight coefficient for the k-th feature attribute; and These are the k-th feature state vectors of target i and target j, respectively.

[0099] The neighborhood radius threshold is dynamically adjusted based on real-time traffic flow density, and adaptive density clustering is performed based on the neighborhood radius threshold and the degree of target correlation. Isolated sensor artifacts and environmental clutter noise in the multi-source heterogeneous traffic data are removed based on the clustering results to obtain clean observation data. Specifically:

[0100] The system dynamically adjusts the neighborhood radius threshold based on real-time traffic flow density. First, the system calculates traffic flow density by counting the number of targets in the current road segment. When traffic is congested and density increases, vehicles are very close together. To prevent misclassifying two adjacent vehicles as one, the system automatically reduces this neighborhood radius threshold. Conversely, when the road is empty, the threshold is appropriately increased to tolerate sensor positioning errors. Combined with an adaptive density clustering mechanism, adaptive density clustering is performed based on the neighborhood radius threshold and the degree of target association (if the target association degree is less than the neighborhood radius threshold, two target points are grouped together). This automatically merges multi-source observation points belonging to the same physical entity into specific association groups. Through this clustering process, the system can effectively identify and eliminate isolated sensor artifacts and environmental clutter noise, providing a clean set of candidate targets for subsequent state estimation.

[0101] S4: By fusing clean observation data from multiple sensors, the optimal fusion estimate of the target's multi-source observations is obtained.

[0102] Within the association group, the system constructs a dynamic uncertainty descriptor to quantitatively evaluate the observation quality of each sensing source, generating a dynamic covariance. This dynamic covariance matrix is ​​generated by acquiring the initial measurement noise covariance matrix of clean observation data and performing real-time dilation correction. Specifically:

[0103] A time decay factor and a distance sensitivity factor were introduced to adjust the initial measurement noise covariance matrix. Perform real-time dilation correction to generate a dynamic covariance matrix. The initial measurement noise covariance matrix is ​​obtained by mapping the precision field in the V2X protocol message.

[0104] This correction process takes into account the decrease in confidence of information over time and with increasing spatial distance, and the mathematical expression is:

[0105]

[0106] in, The process noise covariance matrix represents the basic rate at which the vehicle's position error increases over time when there is no latest observation data. It is a constant matrix derived by discretization of the vehicle's kinematic model or obtained by offline statistical calibration based on real trajectory data. Used to quantify the degree of information loss caused by communication latency; The communication delay is represented by subtracting the generation timestamp of the data packet from the current absolute synchronization timestamp of the system. The distance-sensitive penalty term characterizes the effect of perceived distance. The increased radial and tangential accuracy divergence is addressed by calculating the Euclidean distance using sensor and target coordinates; then, a linear function is used to quantify the confidence level of the distance in data quality. Through this dynamic reassessment mechanism, the system achieves refined perception of the confidence level of heterogeneous data in complex traffic environments.

[0107] Based on the dynamic covariance matrix, an improved dynamic covariance intersection algorithm is used to fuse clean observation data from multiple sensors to obtain the optimal fusion estimate of the target's multi-source observations; specifically:

[0108] The system employs an improved Dynamic Covariance Intersection (ST-CI) algorithm to solve for the optimal fusion estimate of multi-source observations without requiring prior knowledge of the cross-correlation characteristics between sensors. This is achieved by optimizing the weighting coefficients. This results in the optimal state covariance matrix obtained after fusion. The trace is minimized because the trace of the covariance matrix is ​​equivalent to the sum of the variances of the errors of each state variable. Therefore, minimizing this trace physically achieves a strict minimization of the total uncertainty of the multi-source fusion estimation. The core fusion equation is as follows:

[0109]

[0110]

[0111] in, This represents the optimal fusion estimate of multi-source observations of the target. For the first The observation state vector of the target by each sensor. Indicates the first Each sensor weighting coefficient Indicates the first The dynamic covariance matrix of data collected by each sensor This indicates the number of valid multi-source sensor observations that have been successfully captured, matched, and quality-filtered for the target at the current moment.

[0112] The algorithm extracts from the state space The intersection envelope of heterogeneous observation information effectively suppresses the bias caused by obstructed or limited accuracy of single-source sensing. This provides reliable data support for the accurate identification of subsequent abnormal traffic events.

[0113] S5: A heterogeneous graph attention network is used to process the optimal fusion estimation of multi-source observations of the target to obtain the target feature representation; a two-layer stacked LSTM is used to process the target feature representation to obtain the final temporal embedding vector.

[0114] By constructing a heterogeneous graph attention network based on factorized attention, deep decoupling of semantics in traffic scenarios is achieved, such as... Figure 2 As shown, specifically:

[0115] Traffic participants, road topology, and traffic light states are abstracted as graph nodes. A GAT (Graph Attention Network) is used to process the optimal fusion estimation of multi-source observations corresponding to the graph nodes, thereby obtaining the attention weights between the graph nodes. ;

[0116] The target feature representation is calculated based on the attention weights between graph nodes, and the calculation formula is as follows:

[0117]

[0118] in, Represents a node In the Layer feature representation, This represents the activation function. Represents the aggregation coefficient of relation type m. Represents a set of heterogeneous relations. Represents a node For nodes Attention weights This represents the weight matrix of relation type m. Indicates the node at the 1st Layer feature representation In particular, when hour, Represents a node The initial physical semantic feature vector of the input graph network. In the traffic scene graph construction phase, if nodes represent traffic participants, their initial feature vectors... The target's optimal kinematic state, output from the preceding module, is obtained by concatenating and splicing the optimal fusion estimate of multi-source observations and the one-hot encoding of the target type. If a node represents a road topology or a traffic light, its initial features include static geometric boundary coordinates and dynamic phase state. Indicates the heterogeneous relationship type In the topology, with nodes The set of connected neighboring nodes.

[0119] Among them, the aggregation coefficient of the relation type The result is obtained through adaptive computation using a semantic attention mechanism. The specific computation process includes: the system first introduces a learnable global context parameter vector, which is then compared with the node-level attention mechanism (i.e., the computation result inherited from the inner layer of the aforementioned formula) under various heterogeneous relationships. The generated node local features are aggregated and subjected to nonlinear mapping and dot product operations to evaluate the contribution of different relation dimensions to the current traffic semantics. Subsequently, the similarity scores of all relation types are globally normalized using the Softmax activation function, thereby dynamically generating the optimal weight allocation coefficients for different relation types. Based on the above computational architecture, the system can extract and weightedly fuse the following three factorized semantic features in parallel:

[0120] Interaction factors: Modeling physical game and obstacle avoidance behavior between intelligent agents (such as between vehicles, between vehicles and pedestrians);

[0121] Constraint factors: Spatial topological constraints between the modeling agent and the static road topology (such as lane lines and intersection boundaries);

[0122] Rule factor: Modeling the execution logic of intelligent agents on dynamic traffic rules (such as traffic light phases and speed limit instructions).

[0123] In some preferred embodiments of the present invention, in view of the frequent addition and deletion of nodes in traffic flow, the system maintains a global state pool. This state pool uses a dynamic ID indexing mechanism to maintain the hidden state memory of specific targets across consecutive time frames, overcoming the temporal breakage problem caused by dynamic fluctuations of nodes.

[0124] S5: A heterogeneous graph attention network is used to process the optimal fusion estimation of multi-source observations of the target to obtain the target feature representation; a two-layer stacked LSTM is used to process the target feature representation to obtain the final temporal embedding vector.

[0125] A two-layer stacked LSTM is used to capture the long-term evolution features of traffic flow. This architecture enhances feature representation capabilities through dense connections and residual feedback.

[0126]

[0127] In the formula, The original physical feature vector input at the current moment; and These represent the first layer LSTM units in The local hidden state and the long-term memory unit state at any given moment.

[0128] According to the dense fusion operator Combined with cascading splicing operations, the original physical input is... With a layer of semantic representation Deep integration to prevent loss of deep features:

[0129]

[0130] in, The enhanced input vector is generated by the dense fusion operator; Representative feature cascade splicing operator; The activation function is preferred; LeakyReLU activation function is preferred. and They represent dense fusion operators respectively. The corresponding learnable weight matrix and bias vector.

[0131] In higher-order temporal updates, the second-layer LSTM receives composite features and updates the state:

[0132]

[0133] Then it passes through the residual feedback operator The final temporal embedding vector is obtained by constructing multi-scale outputs through element-wise summation. :

[0134]

[0135] This invention employs a deep architecture consisting of two vertically stacked LSTM layers. To balance memory capacity and computational speed, each hidden layer is configured with 128 hidden units. The bottom LSTM is responsible for extracting basic displacement sequence features and passing its 128-dimensional state vector to the second layer for deep logic abstraction. Furthermore, a dropout rate of 0.2 is introduced between layers.

[0136] During model training, the parameters are optimized using an efficient supervised learning strategy. The system employs the Adam optimizer with an initial learning rate of 0.001, executing 50 epochs of complete iterations in batches of 32. During training, the system quantifies the prediction residuals using the cross-entropy loss function and periodically adjusts the weights of each layer using the backpropagation algorithm. To further optimize training efficiency, a dynamic learning rate decay strategy is introduced to ensure smooth convergence of the model as it approaches the optimal solution. Through this streamlined and optimized parameter combination, this invention achieves efficient conversion of perceptual data into event semantics, providing robust logical support for subsequent high-fidelity scene reproduction.

[0137] S6: Input the final temporal embedding vector into the multilayer perceptron for processing to obtain the traffic event type; automatically extract the key abnormal interval data containing the complete evolution logic according to the traffic event type.

[0138] The system inputs a temporal embedding vector into the decision layer of a multilayer perceptron (MLP) and outputs the classification probability of traffic events. The output is a 3-dimensional vector representing the probability of three types of traffic events: normal, violation, and collision.

[0139] If an anomaly occurs, the edge computing node automatically extracts key anomaly interval data containing complete evolution logic and transmits it to the cloud. This means filtering the full data and extracting only the interval data about 10 seconds before and after the traffic incident. The data contains the complete target trajectory and motion state before and after the incident.

[0140] S7: Input key abnormal interval data into the virtual engine to reproduce the entire process of the traffic incident.

[0141] like Figure 3 As shown, the structured data parsing program is started to convert key parameters such as pose, velocity, and heading in the critical anomaly interval data into state vectors that the virtual engine can recognize. The virtual engine then recreates the entire traffic event process; specifically:

[0142] The system synchronously accesses external map data and enters the object modeling stage. After pre-modeling the objects to be generated in 3D, this stage is further decoupled into two branches: static object modeling and dynamic object modeling. The former uses map topology and roadside infrastructure information to construct an accurate geometric environment, that is, automatically constructing a static traffic scene with real geographical attributes in Unreal Engine. The latter retrieves and instantiates the corresponding 3D models of vehicles, pedestrians, etc. from the resource pool based on the parsed physical features of the entities, realizing the transformation of physical entities into virtual objects.

[0143] By constructing a virtual-real mapping interface, an association mapping is established between structured data packages and Unreal Engine virtual scene objects. For different types of traffic elements, the system instantiates corresponding Actor objects in the virtual scene: for static elements, the interface performs a one-time spatial pose mapping to establish environmental constraints; for dynamic objects, the system retrieves the corresponding 3D model from the resource pool based on a globally unique ID and updates its spatial state in real time by calling underlying components.

[0144] A time scheduling mechanism based on dual-clock synchronization is adopted, and the simulation logic time is defined as follows: The linear mapping relationship between it and the real-time data is as follows:

[0145]

[0146] in, For time-flow rate factor, To reproduce the initial reference time, This indicates the current real-world physics system time in the rendering engine. This represents the initial real-world physical system time when the replay task is initiated. Within each scheduling cycle, the system calculates the data frame index to be extracted using the following formula. :

[0147]

[0148] in, This index represents the fixed time interval between two adjacent frames of observation data in the data cache sequence. Using this index, the system retrieves the traffic participant state vector corresponding to the current moment from the data cache, thereby aligning the simulation process with the physical time evolution logic.

[0149] To adapt discrete sampling data to continuous frame updates from the engine, the system employs a quaternion spherical linear interpolation algorithm to smooth the rotational attitude. Let the current frame attitude quaternion be... The target time quaternion is The interpolation step size is Smooth posture The calculation process is as follows:

[0150]

[0151] Interpolation weights From angular velocity gain coefficient With time step Decide:

[0152]

[0153] Smoothing ensures the continuity of movement trajectories for traffic participants in virtual space, eliminating visual jitter.

[0154] A three-tiered index architecture of "event anchor point - timestamp - object ID" is constructed. The first-level index uses exception events as logical units, encapsulating event metadata. , Indicates the type of abnormal traffic incident. Indicates the end reference time for reproduction. This represents the set of unique identifiers for all traffic entities involved in the event. This describes the core physical spatial location or coordinates where the event occurred. The secondary index uses fixed-length slicing technology to divide the event duration into segments. Equal-length blocks, block index The calculation is as follows:

[0155]

[0156] The third-level index uses the ID as the key to build a dynamic lookup table.

[0157] Within Unreal Engine's Tick cycle, the system constructs a full-scene state vector matrix and ensures that multiple subjects are updated within the same rendering cycle through synchronous execution of operators. The Tick function accesses the state cache via iterators and calls the underlying mapping interface. :

[0158]

[0159] in, This is the pose vector after smoothing calculation. Represents the three-dimensional spatial coordinates of traffic participants in a virtual scene. This represents the three-dimensional rotational attitude angle of the model. This represents the model's additional dynamic attributes and animation states. This process enables the synchronous evolution of vehicles, pedestrians, and environmental elements within a virtual scene, achieving a dynamic reproduction of the entire traffic event process.

[0160] The above-described embodiments further illustrate the purpose, technical solution, and advantages of the present invention. It should be understood that the above-described embodiments are merely preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made to the present invention within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for digital twin reconstruction of traffic events based on vehicle-road-cloud perception data, characterized in that, Includes the following steps: S1: Under the vehicle-road-cloud collaborative perception architecture, collect multi-source heterogeneous traffic data and unify the spatial coordinate system of multi-source heterogeneous traffic data; S2: Spatiotemporal alignment of multi-source heterogeneous traffic data after unifying the spatial coordinate system; S3: Remove isolated sensor artifacts and environmental clutter noise from multi-source heterogeneous traffic data to obtain clean observation data; S4: By fusing clean observation data from multiple sensors, the optimal fusion estimate of the target's multi-source observations is obtained; S5: A heterogeneous graph attention network is used to process the optimal fusion estimation of multi-source observations of the target to obtain the target feature representation; a two-layer stacked LSTM is used to process the target feature representation to obtain the final temporal embedding vector; S6: Input the final temporal embedding vector into a multilayer perceptron for processing to obtain the traffic event type; automatically extract key abnormal interval data containing complete evolution logic based on the traffic event type; S7: Input key abnormal interval data into the virtual engine to reproduce the entire process of the traffic incident.

2. The method for digital twin reconstruction of traffic events based on vehicle-road-cloud perception data according to claim 1, characterized in that, In step S1, the process of collecting multi-source heterogeneous traffic data includes: The vehicle unit on board the intelligent connected vehicle acquires the vehicle's motion state, position coordinates and dynamic parameters in real time, and encapsulates them into standardized basic safety messages; Using roadside units and their associated visual and radar sensing equipment, the dynamic spatiotemporal trajectories of other traffic participants in the scene are detected and captured, and encapsulated as sensing sharing messages. The system collects phase status and countdown data from roadside traffic signal controllers in real time and integrates road topology key points, lane flow semantics, and geometric and physical constraint information provided by commercial high-precision maps.

3. The method for digital twin reconstruction of traffic events based on vehicle-road-cloud perception data according to claim 1, characterized in that, Step S2 specifically includes: Calculate the absolute time offset of multi-source heterogeneous traffic data; Multiple kinematic models are run in parallel, and the outputs of the multiple kinematic models are weighted and fused according to the absolute time offset to obtain the final calibrated pose state of the target; the kinematic models include a uniform linear kinematic model, a uniformly accelerated linear kinematic model, and a constant rotation kinematic model.

4. The method for digital twin reconstruction of traffic events based on vehicle-road-cloud perception data according to claim 3, characterized in that, The uniformly accelerated linear kinematic model is represented as follows: ； in, and These represent the initial raw data of the target at the moment of perception data generation. coordinates and coordinate, Indicates calibration delay. Indicates the instantaneous velocity of the target. Indicates the heading angle of the target. Indicates the target's acceleration. This indicates the alignment position after kinematic model derivation; This represents the instantaneous velocity of the target after kinematic model derivation; The uniform linear kinematic model is represented as follows: ； in, and These represent the target at the observation time. coordinates and coordinate, This represents the initial instantaneous value of the kinematic parameters at the moment the sensor data is generated; The constant rotation kinematic model is expressed as: ； in, Represents the angular velocity of the target. This represents the heading angle of the target after the kinetic model derivation.

5. The method for digital twin reconstruction of traffic events based on vehicle-road-cloud perception data according to claim 3, characterized in that, The weighted fusion of the outputs from multiple kinematic models is represented as follows: ； in, Indicates the target is The final calibrated pose state after multi-model weighting at each moment. Indicates the first A kinematic model in The state estimation vector at each moment, Indicates the number of motion models. Indicates the first The posterior probability weights of each sub-model at the current time step This represents the time transition matrix of the corresponding model; This represents the absolute time offset between the data timestamp and the system master clock.

6. The method for digital twin reconstruction of traffic events based on vehicle-road-cloud perception data according to claim 1, characterized in that, Step S3 specifically includes: Extract the target's instantaneous position, velocity vector, heading angle, and geometric dimensions, and construct a joint feature state vector; Calculate the degree of target correlation based on the joint feature state vector; The neighborhood radius threshold is dynamically adjusted based on real-time traffic flow density, and adaptive density clustering is performed based on the neighborhood radius threshold and the degree of correlation with the target. Based on the clustering results, isolated sensor artifacts and environmental clutter noise in multi-source heterogeneous traffic data are removed to obtain clean observation data.

7. The method for digital twin reconstruction of traffic events based on vehicle-road-cloud perception data according to claim 1, characterized in that, Step S4 specifically includes: The initial measurement noise covariance matrix of the clean observation data is obtained and real-time dilation correction is performed to generate a dynamic covariance matrix; Based on the dynamic covariance matrix, an improved dynamic covariance intersection algorithm is used to fuse clean observation data from multiple sensors to obtain the optimal fusion estimate of the target's multi-source observations.

8. A method for digital twin reconstruction of traffic events based on vehicle-road-cloud perception data according to claim 7, characterized in that, The improved dynamic covariance intersection algorithm is used to fuse clean observation data from multiple sensors, as follows: ； ； in, This represents the optimal fusion estimate of multi-source observations of the target. For the first The observation state vector of the target by each sensor. Indicates the first Each sensor weighting coefficient Indicates the first The dynamic covariance matrix of data collected by each sensor Indicates the number of effective multi-source sensors. This represents the optimal covariance matrix of the target after weighted fusion of multi-source observations.

9. A method for digital twin reconstruction of traffic events based on vehicle-road-cloud perception data according to claim 1, characterized in that, In step S5, the process of using a heterogeneous graph attention network to process the optimal fusion estimation of multi-source observations of the target includes: Traffic participants, road topology, and traffic light status are abstracted as graph nodes; The GAT network is used to process the optimal fusion estimation of multi-source observations corresponding to graph nodes to obtain the attention weights between graph nodes; The target feature representation is calculated based on the attention weights between graph nodes, and the calculation formula is as follows: ； in, Represents a node In the Layer feature representation, This represents the activation function. Represents the aggregation coefficient of relation type m. Represents a set of heterogeneous relations. Represents a node For nodes Attention weights This represents the weight matrix of relation type m. Indicates the node at the 1st Layer feature representation , Indicates the heterogeneous relationship type In the topology, with nodes The set of connected neighboring nodes.

10. A method for digital twin reconstruction of traffic events based on vehicle-road-cloud perception data according to claim 1, characterized in that, In step S5, the target feature representation is processed using a double-layer stacked LSTM, as follows: ； ； ； ； in, Represents the final temporal embedding vector. and These represent the first LSTM layer in... The hidden state and memory unit state at any given moment. This represents the first layer LSTM cell. This represents the target feature representation input at the current time. and These represent the first layer LSTM units in... The local hidden state and memory unit state at any given moment. This represents the augmented input vector. This represents the activation function. and They represent dense fusion operators respectively. The learnable weight matrix and bias vector, This represents the feature concatenation operator. and These represent the second LSTM layer in... The intermediate hidden states and memory unit states at any given moment. This indicates the second-layer LSTM unit. and These represent the second LSTM layer in... The hidden state and memory unit state at any given moment. This represents the residual feedback operator.

Images