A highly integrated fusion perception system based on road traffic safety facilities

Abstract

This invention relates to the field of road vehicle perception, specifically a highly integrated fusion perception system based on road traffic safety facilities. It includes a multimodal fusion module: performing spatiotemporal synchronization and deep fusion of heterogeneous raw data to generate a dynamic traffic situation map; an interaction relationship cognition module: constructing a spatiotemporal interaction relationship graph; outputting structured interaction relationships through an inference model; a prediction and monitoring module: fusing the dynamic traffic situation map from the structured interaction relationships through a time-series prediction model to generate traffic prediction data; when the potential traffic conflict risk exceeds a preset risk threshold, it is encapsulated as critical traffic event data, encrypted, and uploaded; and a cloud-based offline optimization module: performing adaptive optimization using the critical traffic event data; and conducting stress testing and safety verification. This invention improves active safety by constructing an intelligent traffic safety system.

Description

Technical Field

[0001] This invention relates to the field of road vehicle perception, specifically to a highly integrated fusion perception system based on road traffic safety facilities. Background Technology

[0002] As traffic congestion and safety pressures intensify, intelligent transportation systems are beginning to build road network perception capabilities through the widespread deployment of sensors such as cameras and radar. In this process, data fusion is a key means to break through the physical limitations of single sensors and reconstruct the complete traffic scene. With the help of the advancements in vehicle-road cooperation and collaborative perception technologies in recent years, the creation of a real-time and blind-spot-free global perception network is gradually becoming a reality and a core path to improve traffic safety.

[0003] However, current sensing technologies in the field of intelligent transportation still exhibit some limitations in application. Regarding the depth of sensing fusion, many systems focus on analyzing data from single sources or performing basic "sensor-response" linkages. Furthermore, some systems rely on historical data models for risk prediction, which limits their response speed and accuracy when dealing with dynamic risks caused by sporadic, non-patterned factors. In addition, how to effectively organize multiple dispersed sensing nodes to form a macroscopic, collaborative understanding of traffic conditions over a wider area, and achieve cross-regional, multi-device information sharing and collaborative work, is still in the exploratory stage. In summary, the current challenge is how to construct a comprehensive road environment sensing network with forward-looking predictive capabilities.

[0004] To address this, a highly integrated fusion perception system based on road traffic safety facilities is proposed. Summary of the Invention

[0005] The purpose of this invention is to provide a highly integrated fusion perception system based on road traffic safety facilities to solve the problems mentioned in the background art.

[0006] To achieve the above objectives, the present invention provides the following technical solution: a highly integrated fusion perception system based on road traffic safety facilities, comprising: Multimodal fusion module: performs spatiotemporal synchronization and deep fusion of heterogeneous raw data from traffic safety facilities to generate dynamic traffic situation maps; Interaction Relationship Recognition Module: Constructs a spatiotemporal interaction relationship diagram of the traffic scenario through the dynamic traffic situation map, and infers the spatiotemporal interaction relationship diagram through the reasoning model to output structured interaction relationships; Prediction and monitoring module: Extracts dynamic interaction features from the structured interaction relationship, fuses the dynamic traffic situation map through a time-series prediction model, and generates traffic prediction data containing driving trajectories and potential traffic conflict risks; monitors the traffic prediction data, and when the potential traffic conflict risk exceeds a preset risk threshold, encapsulates the dynamic traffic situation map, structured interaction relationship, and traffic prediction data into critical traffic event data, and uploads it in encryption. Cloud-based offline optimization module: Adaptively optimizes the time-series prediction model using the critical traffic event data; generates test scenarios through a simulation platform, conducts stress tests and safety verifications, and obtains an update package.

[0007] Preferably, the specific process of the multimodal fusion module includes: performing time alignment based on a unified timestamp on heterogeneous raw data from traffic safety facilities including cameras, lidar, and millimeter-wave radar, and achieving spatial alignment through coordinate system transformation; extracting intermediate layer feature vectors from the aligned image data, point cloud data, and radar data respectively using a deep learning network; weighting and fusing the intermediate layer feature vectors to generate a feature representation under a unified bird's-eye view, and generating the dynamic traffic situation map based on the feature representation.

[0008] Preferably, the specific process of the interaction relationship cognition module includes: identifying traffic participants from the dynamic traffic situation map, treating each traffic participant as a graph node; establishing graph edges between graph nodes with potential interactive behaviors based on the spatial proximity and motion vector relationships between the traffic participants, jointly constructing the spatiotemporal interaction relationship graph; extracting the dynamic attributes of each graph node, including position, speed, acceleration, and heading angle, as node features; inputting the spatiotemporal interaction relationship graph with node features into the inference model, updating the node features through the message passing and aggregation mechanism of the inference model to integrate the interactive influence of neighboring nodes; decoding the updated node features, outputting the structured interaction relationship, which includes quantified vehicle following, lane changing intentions, and pedestrian-vehicle conflict risks.

[0009] Preferably, the specific generation process of the traffic prediction data includes: extracting historical motion state sequences for each traffic participant in the dynamic traffic situation map, and extracting dynamic interaction features of quantified interaction relationships from the structured interaction relationships; concatenating the historical motion state sequences with the dynamic interaction features, and integrating the spatial features of the dynamic traffic situation map to construct a temporal input feature vector for each traffic participant; inputting the temporal input feature vector into the temporal prediction model, and having the temporal prediction model output a sequence of position coordinates of the traffic participants at multiple future times to form a predicted driving trajectory; assessing the potential traffic conflict risk by calculating the spatiotemporal overlap probability of the predicted driving trajectories of different traffic participants; and merging the predicted driving trajectories of all traffic participants with the potential traffic conflict risks to generate the traffic prediction data.

[0010] Preferably, the specific process of extracting dynamic interaction features from the structured interaction relationship includes: identifying the specific type of the structured interaction relationship, wherein the type includes a preset discrete interaction event type and a target-associated interaction state type; when the type is a discrete interaction event type, generating a one-hot encoded vector representing the event classification as the dynamic interaction feature; when the type is a target-associated interaction state type, extracting the relative distance, relative speed, and relative acceleration of the associated target traffic participants to form a target state feature vector as the dynamic interaction feature.

[0011] Preferably, the specific generation process of the critical traffic event data includes: streaming monitoring of potential traffic conflict risks in the traffic prediction data, continuously comparing the quantified value of the potential traffic conflict risk with a preset risk threshold; when the quantified value exceeds the risk threshold, triggering an event recording program to capture the dynamic traffic situation map, the structured interaction relationship, the traffic prediction data at the trigger time, and the traffic evolution data within a preset time window after the trigger time within a preset time window; assigning a unique event identifier and timestamp to the captured data set, integrating it into structured critical traffic event data; encrypting the critical traffic event data using an asymmetric encryption algorithm, and uploading the encrypted data to a cloud server through a secure transmission protocol.

[0012] Preferably, the specific process of adaptive optimization includes: dividing the received critical traffic event data into a training set and a validation set, and performing data augmentation on the training set by perturbing the initial state of traffic participants to generate diverse derivative risk scenarios; extracting original traffic prediction data and traffic evolution data from each data point in the training set, calculating the prediction error as a loss function; using the loss function, fine-tuning and updating the network weights of the time-series prediction model through a backpropagation algorithm; and using the validation set to evaluate the performance of the updated time-series prediction model.

[0013] Preferably, the specific process of stress testing and safety verification includes: reconstructing the critical traffic event data in the simulation platform, and generating a series of edge test scenarios by perturbing the key parameters in the scenarios; comparing the optimized time-series prediction model with the unoptimized time-series prediction model in the edge test scenarios to evaluate key safety indicators, including prediction accuracy, false negative rate, and false alarm rate; and when the key safety indicators of the optimized time-series prediction model are better than the preset verification standards, encapsulating them into an encrypted update package.

[0014] Compared with the prior art, the beneficial effects of the present invention are as follows: 1. The system uses a multimodal traffic situation map to build a spatiotemporal interaction relationship map, which can identify traffic participants in the scene. The core is to deduce the interaction behaviors that are happening or about to happen between targets. After clarifying these dynamic relationships, the subsequent trajectory prediction has a more objective basis in the face of dense and complex traffic flow.

[0015] 2. The interaction relationships extracted in the preceding sequence will be treated as dynamic features and directly integrated into the time series prediction model. In this way, the prediction process no longer focuses only on the historical trajectory of a single vehicle, but also takes into account the mutual influence and game intentions of surrounding vehicles. This approach is closer to the real driving logic and can expose the potential conflict risks brought about by multi-vehicle interactions earlier, thus gaining more reaction time for traffic control and early warning.

[0016] 3. At the self-optimization level, the prediction and monitoring module automatically collects data on high-risk critical events in actual road conditions; the cloud-based offline module uses these real-world difficult case samples to fine-tune the prediction model. By establishing this closed-loop process, the system can perform regular iterations based on continuously accumulated road environment data, gradually improving the overall perception and prediction effects. Attached Figure Description

[0017] Figure 1 This is a structural diagram of a highly integrated fusion perception system based on road traffic safety facilities, as proposed in an embodiment of this invention. Figure 2This is a flowchart of the real-time edge sensing and prediction process proposed in an embodiment of the present invention. Figure 3 This is a flowchart illustrating the optimization and deployment process of the cloud-based closed-loop model proposed in an embodiment of this invention. Detailed Implementation

[0018] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0019] Please see Figures 1 to 3 The present invention provides a highly integrated fusion perception system based on road traffic safety facilities, the specific modules of which are as follows: Multimodal fusion module: performs spatiotemporal synchronization and deep fusion of heterogeneous raw data from traffic safety facilities to generate dynamic traffic situation maps; Interaction Relationship Recognition Module: Constructs a spatiotemporal interaction relationship diagram of the traffic scenario through the dynamic traffic situation map, and infers the spatiotemporal interaction relationship diagram through the reasoning model to output structured interaction relationships; Prediction and monitoring module: Extracts dynamic interaction features from the structured interaction relationship, fuses the dynamic traffic situation map through a time-series prediction model, and generates traffic prediction data containing driving trajectories and potential traffic conflict risks; monitors the traffic prediction data, and when the potential traffic conflict risk exceeds a preset risk threshold, encapsulates the dynamic traffic situation map, structured interaction relationship, and traffic prediction data into critical traffic event data, and uploads it in encryption. Cloud-based offline optimization module: Adaptively optimizes the time-series prediction model using the critical traffic event data; generates test scenarios through a simulation platform, conducts stress tests and safety verifications, and obtains an update package.

[0020] The technical solution of the present invention will be further described in detail below with reference to specific embodiments.

[0021] Example 1 This application discloses a highly integrated fusion perception system based on road traffic safety facilities. When a vehicle passes through intersection A, this system begins collecting data through the safety facilities. (See also...) Figure 1The specific modules proposed in this invention include: a multimodal fusion module: performing spatiotemporal synchronization and deep fusion of heterogeneous raw data from traffic safety facilities to generate a dynamic traffic situation map; an interaction relationship cognition module: constructing a spatiotemporal interaction relationship map of traffic scenarios through the dynamic traffic situation map, and inferring the spatiotemporal interaction relationship map through an inference model to output structured interaction relationships; a prediction and monitoring module: extracting dynamic interaction features from the structured interaction relationships, fusing the dynamic traffic situation map through a time-series prediction model to generate traffic prediction data containing driving trajectories and potential traffic conflict risks; monitoring the traffic prediction data, and when the potential traffic conflict risk exceeds a preset risk threshold, encapsulating the dynamic traffic situation map, structured interaction relationships, and traffic prediction data into critical traffic event data, and encrypting and uploading it; a cloud-based offline optimization module: adaptively optimizing the time-series prediction model using the critical traffic event data; generating test scenarios through a simulation platform, conducting stress tests and security verification, and obtaining an update package.

[0022] Furthermore, heterogeneous raw data from traffic safety facilities are spatiotemporally synchronized and deeply fused to generate a dynamic traffic situation map; the specific implementation process includes: For heterogeneous raw data from traffic safety facilities including cameras, lidar, and millimeter-wave radar, time alignment is performed based on a unified timestamp, and spatial alignment is achieved through coordinate system transformation. A deep learning network is used to extract intermediate layer feature vectors from the aligned image data, point cloud data, and radar data, respectively. The intermediate layer feature vectors are weighted and fused to generate a feature representation under a unified bird's-eye view, and the dynamic traffic situation map is generated based on the feature representation.

[0023] Specifically, the process of the multimodal fusion module is as follows: The system's perception data is primarily provided by sensor matrices on traffic light poles at intersections. To ensure consistency in subsequent data processing, it is first necessary to clarify the input formats and dimensions of various sensors: Image data is acquired by cameras, including aspect ratio, resolution, and RGB color channels. Its frame rate setting references the basic principle of visual persistence, ensuring clear capture of fast-moving targets while controlling the overall data throughput of the system; Point cloud data consists of disordered points and their characteristics (3D coordinates and reflectivity) acquired by single-frame scans from LiDAR. Moreover, due to limitations imposed by the physical rotation cycle of mechanical or semi-solid-state radar motors, the sampling frequency of this type of data is often relatively fixed; Millimeter-wave radar data covers the target's 3D coordinates, radial velocity, and radar cross section, primarily used to provide higher-frequency velocity status updates than LiDAR.

[0024] Considering the differences in hardware clocks and sampling rates among different sensors, in order to maintain time synchronization between each sensor and the edge computing unit, the system introduces a precise time protocol. In specific data alignment operations, the LiDAR with the lowest sampling rate is usually used as the main trigger reference: whenever a new frame of point cloud data is received, the system will directly go to the image and millimeter-wave radar buffers and select the data frame with the closest timestamp for matching.

[0025] System-defined time tolerance threshold (The unit is seconds), the basis for selecting this parameter is, ,in, This represents the maximum allowable physical position drift tolerance distance (in meters) for the target detection algorithm. The maximum speed limit for the road in the application scenario (in meters per second); if the absolute time difference is less than this threshold, the data is considered to be successfully aligned.

[0026] Spatial alignment transforms all sensor data into a unified reference world coordinate system. In this embodiment, a reference three-dimensional Cartesian coordinate system is established with the optical center of the intersection lidar as the origin.

[0027] For the camera, the intrinsic parameter matrix of the camera and the extrinsic parameter matrix (including rotation and translation vectors) of the camera relative to the LiDAR are obtained through offline calibration. For any point in space, the homogeneous coordinates in the LiDAR coordinate system are... Its projection onto the image pixel plane coordinates The mathematical relationships for coordinate system transformation are as follows: ; In this formula, the variables are explained as follows: x, y, z are the three-dimensional physical coordinates of the spatial point in the lidar reference coordinate system; s is the depth scale factor, which represents the depth value of the spatial point in the direction of the camera optical axis and is used to normalize the homogeneous coordinates; u and v represent the horizontal and vertical pixel coordinates on the image plane, respectively, and their values ​​are limited by the pixel width and height of the image. This is the extrinsic transformation matrix, used to transform points in the lidar coordinate system to the camera coordinate system; It includes the camera's focal length and principal point offset, used to project 3D points in the camera coordinate system onto a 2D pixel plane.

[0028] Millimeter-wave radar data has three-dimensional coordinate attributes, which can be directly transformed into the lidar coordinate system through a rigid body transformation matrix that includes rotation and translation.

[0029] After completing the spatiotemporal alignment, the system uses a deep learning network to extract intermediate layer feature vectors from image data, point cloud data, and radar data, respectively.

[0030] For intermediate layer feature extraction of point cloud data, the system adopts the PointPillars architecture to process point cloud data. This architecture does not use 3D convolution and adapts to the deployment limitations of edge computing devices. First, the monitoring space range of the roadside equipment is defined. The physical range boundary values ​​of the X and Y axes are selected as follows: greater than or equal to half of the actual maximum diagonal length of the deployed intersection. The range of the Z axis is selected as follows: the lower limit is lower than the lowest elevation of the road surface in the monitoring area, and the upper limit is higher than the maximum height of common heavy vehicles and lower than the ground clearance of the overhead traffic facilities at the intersection. The selection of this range is based on covering the boundary of a standard urban intersection and filtering out useless echoes in the air.

[0031] The space is divided into vertical cylinders. The length and width resolution R of a single cylinder is chosen to be half the average physical width of the smallest target that the system needs to identify. This resolution can preserve the spatial geometric details of small targets such as pedestrians while preventing the generated feature mesh from becoming too sparse.

[0032] A simplified PointNet network is used to encode all points falling within the same cylinder using a nonlinear multilayer perceptron to extract the geometric features of the local region. The extracted features are then scattered back to the original two-dimensional grid location, forming pseudo-image features. The spatial dimension N of this two-dimensional grid is derived from a formula. Based on the aforementioned physical range and The grid size is calculated using the resolution R. Finally, a two-dimensional convolutional layer is used to output the intermediate feature vector of the point cloud.

[0033] For intermediate layer feature extraction of image data, the system uses ResNet-50 as the backbone network to extract multi-scale visual features. After processing, the output feature map of Conv4 layer is retained, which has an original dimension of 512 channels. In order to align with the dimension of point cloud features and reduce the computational burden, it is reduced to 64 channels through pointwise convolution operation.

[0034] Considering that the image is from a perspective view while the point cloud is from a bird's-eye view, there are differences in their spatial representations. Therefore, it is necessary to first predict the discrete depth distribution of each pixel in the image, calculate the truncated cone features with depth probability, and then combine the camera's intrinsic and extrinsic parameter calibration data to map and pool these features onto a preset bird's-eye view grid to generate the intermediate layer feature vector of the image.

[0035] For millimeter-wave radar data, the processing flow is to first use a multilayer perceptron to directly encode its three-dimensional coordinates, radial velocity, and reflection cross section, and then project it into the same bird's-eye view grid through coordinate scattering operations to generate radar feature vectors with consistent dimensions.

[0036] After generating feature vectors for the three modalities, the system uses a combination of tensor concatenation and two-dimensional convolution to weight and fuse the intermediate layer feature vectors. The mathematical calculation process is as follows: ; In the formula, This indicates tensor splicing along the channel dimension; This represents element-wise multiplication; These are the intermediate layer feature vectors for point cloud, image, and millimeter-wave radar, respectively, with their tensor dimensions all aligned to 64×625×625; These are the attention weight matrices for the corresponding modalities, which are learned by the network from historical data through the backpropagation algorithm. Their tensor dimensions are consistent with the corresponding intermediate layer feature vectors. After channel concatenation, an intermediate tensor with 192 channels is formed. This represents a 2D convolution operation with a kernel size of 3×3, used for channel dimensionality reduction and deep cross-modal feature fusion of the concatenated feature tensor; The output dimension of the fused integrated feature tensor is reduced to 64×625×625.

[0037] The fused features are scaled through a feature pyramid network and then input into the detection head to complete target classification and bounding box regression.

[0038] The output data of the multimodal fusion module is a dynamic traffic situation map, which contains fused feature tensors. And the set of targets of all traffic participants parsed at the current moment, each target including its three-dimensional coordinates, size, category label and confidence score.

[0039] By extracting and performing attention-weighted fusion through deep learning networks, the complementary information of each sensor can be preserved to the maximum extent. This allows for the generation of high-quality, information-rich bird's-eye view dynamic traffic situation maps even in scenarios where the performance of a single sensor is degraded, such as in adverse weather or poor lighting conditions.

[0040] Furthermore, a spatiotemporal interaction relationship diagram of the traffic scenario is constructed using the dynamic traffic situation map; the spatiotemporal interaction relationship diagram is inferred using an inference model to output structured interaction relationships; the specific process includes: Traffic participants are identified from the dynamic traffic situation map, and each participant is treated as a graph node. Based on the spatial proximity and motion vector relationships between these participants, edges are established between graph nodes with potential interactive behaviors to jointly construct the spatiotemporal interaction graph. Dynamic attributes of each graph node, including position, speed, acceleration, and heading angle, are extracted as node features. The spatiotemporal interaction graph with node features is input into the inference model. Through the message passing and aggregation mechanism of the inference model, the node features are updated to integrate the interactive influence of neighboring nodes. The updated node features are decoded to output the structured interaction relationship, which includes quantified vehicle following, lane-changing intentions, and pedestrian-vehicle conflict risks.

[0041] Specifically, the process of the interaction relationship cognition module is as follows: From the input dynamic traffic situation map, the system traverses each traffic participant and abstracts it as a graph node in the spatiotemporal interaction relationship graph.

[0042] When extracting node features, the system converts the various dynamic attributes of traffic participants into node feature vectors. These vectors contain eight elements: the target's absolute lateral and longitudinal coordinates in the bird's-eye view coordinate system, the current absolute lateral and longitudinal velocities, the lateral and longitudinal accelerations calculated by the difference between adjacent time steps, the heading angle (in radians) with due north as the reference, and a one-hot encoded scalar value that distinguishes the target category (e.g., 1 for motor vehicles and 2 for pedestrians and non-motor vehicles).

[0043] The system establishes the edges of the graph based on the spatial proximity and motion vector relationship between traffic participants. For any two nodes, the system calculates the relative distance between them, which is equal to the square root of the sum of the squares of the difference in the horizontal coordinates and the squares of the difference in the vertical coordinates of the two nodes.

[0044] The system calculates a dynamic perception radius threshold based on the vehicle's driving status. The selection method for this dynamic perception radius threshold is as follows: ; in The basic security concern constants set for the system; The combined reaction time of the driver and the system; u is the road friction coefficient; g is the acceleration due to gravity; This represents the absolute velocity of the current node; in a typical urban straight and dry road surface scenario, to reduce the nonlinear computational overhead of edge computing devices, this embodiment retains the reaction distance term (i.e., The original linear form of the braking distance term is obtained by performing piecewise linear fitting on the nonlinear physical formula related to the square of the vehicle speed within the common speed limit range of the system to obtain approximate calculation coefficients. In addition, the basic safety concern range constant is set to 20 meters as the lower limit protection value of the dynamic perception radius when the vehicle is in a low-speed or stationary scenario.

[0045] The system will only establish a directed edge between two nodes when the distance between them is less than the dynamic sensing radius and the angle between their directions of movement is less than a preset direction angle threshold. This direction angle threshold is typically the complementary angle of the maximum angle between the centerlines of lanes that may intersect within the intersection. This condition is set to filter out traffic participants traveling in opposite directions whose trajectories will not substantially intersect, thus avoiding invalid connections.

[0046] After extracting node features and filtering edge connection conditions, the system constructs the spatiotemporal interaction graph at the current moment, and finally outputs a complete set of nodes, edges, and feature matrix.

[0047] A spatiotemporal interaction graph with node features is input into the inference model. This embodiment employs a spatiotemporal graph attention network. Through the message passing and aggregation mechanism of this inference model, the network updates the features of each node to incorporate the influence from neighboring nodes.

[0048] For node i, its feature vector in the l-th graph convolutional layer It is based on its first-order neighborhood node set The features are updated using the following mathematical formula: ; Explanation of the meaning of variables in the calculation formula and instructions for their recurrence: and : Represents the hidden feature vectors of node i in layer l and layer l+1, respectively; D is the dimension of the hidden layer features of the network; when l is 0, it is the initial node feature vector; The learnable linear transformation weight matrix of the l-th layer is used to project the input node features onto the latent semantic space. Its tensor dimension is D×8 for the input layer and D×D for the subsequent hidden layers. : Non-linear activation function. This embodiment uses an exponential linear unit, which contains a non-zero gradient for negative inputs, maintaining network parameter updates when node features exhibit negative coordinates or negative velocities; The set of neighboring nodes of node i, including node i itself; Attention weight coefficient: This represents the numerical value of the interaction influence exerted by node j on node i. Its calculation formula is as follows: ; In this formula, For the weight vector of a single-layer feedforward neural network, its tensor dimension is strictly set to 2D×1 in order to achieve dimensionality consistency and output scalar attention weights; This represents concatenating two vectors along their feature dimensions. The formula uses a normalized exponential function to ensure that the sum of the weights of all neighboring nodes for node i is 1.

[0049] In terms of network structure design, the number of graph convolutional layers is set to 3, mainly to meet the system's requirement to capture the highest third-order interaction transit between targets (i.e., third-order topological neighborhood); at the same time, the model introduces a multi-head attention mechanism containing 4 independent computation heads, and outputs the independent results of each computation head after splicing them together.

[0050] After these three layers of message passing and aggregation operations, the node features are further integrated with contextual information. Then, the system uses two fully connected layers to map and decode the features of these updated target nodes. The decoder will finally output structured interaction relationship data, which includes the following three items: first, a quantitative probability score of the following intention between vehicles; second, a three-dimensional lane change probability distribution vector reflecting the intention to turn left, go straight, and turn right; and third, a quantitative score of human-vehicle conflict risk specifically calculated for heterogeneous node pairs such as motor vehicles and pedestrians or non-motor vehicles.

[0051] Before actual system deployment, the inference model needs to undergo basic training through offline supervised learning. The training dataset contains spatiotemporal interaction graphs extracted from historical traffic scenarios and their corresponding ground truth labels. Specifically, the ground truth labels for vehicle following intentions are marked based on whether the headway between the following and preceding vehicles remains within a stable threshold within a preset timeframe in the historical trajectory; lane-changing intentions are marked based on the actual lane-changing actions and crossing times; and pedestrian-vehicle conflict risk is regressed and labeled based on an objective risk coefficient converted from the actual minimum distance and relative speed. During training, the cross-entropy loss function is used for the classification outputs of following and lane-changing intentions, while the mean squared error loss function is used for the quantitative scoring output of pedestrian-vehicle conflict risk. The model updates the weight parameters of the network graph convolutional and fully connected layers through backpropagation until the loss value on the validation set converges, resulting in an initial model with usable inference capabilities.

[0052] By abstracting traffic scenarios into spatiotemporal interaction graphs and using graph neural networks for reasoning, the system captures and quantifies dynamic and complex interactive behaviors, enabling it to understand the logic and intent behind driving behavior and improving its ability to identify potential risks in complex, high-density traffic flows.

[0053] Furthermore, dynamic interaction features are extracted from the structured interaction relationships, and the dynamic traffic situation map is fused through a time-series prediction model to generate traffic prediction data containing driving trajectories and potential traffic conflict risks; see reference. Figure 2The specific implementation process includes: For each traffic participant in the dynamic traffic situation map, a historical motion state sequence is extracted, and dynamic interaction features of quantified interaction relationships are extracted from the structured interaction relationships. The historical motion state sequence and the dynamic interaction features are concatenated and integrated with the spatial features in the dynamic traffic situation map to construct a temporal input feature vector for each traffic participant. The temporal input feature vector is input into the temporal prediction model, which outputs a sequence of position coordinates of the traffic participants at multiple future times to form a predicted driving trajectory. By calculating the spatiotemporal overlap probability of the predicted driving trajectories of different traffic participants, the potential traffic conflict risk is assessed. The predicted driving trajectories of all traffic participants are merged with the potential traffic conflict risk to generate the traffic prediction data.

[0054] Specifically, the process of generating traffic prediction data is as follows: The module extracts dynamic interaction features of quantitative interaction relationships from structured interaction relationships. The system identifies the specific types of structured interaction relationships, which are divided into discrete interaction event types and target-related interaction state types.

[0055] When the type is a discrete interactive event, the system generates a one-hot encoded vector representing the event category as a dynamic interactive feature. In this embodiment, a vector consisting of four Boolean values ​​is configured internally, corresponding to four fixed categories: no special event, emergency braking, illegal lane change, and forced cutting in.

[0056] When a certain type of event occurs, the corresponding position is assigned the value 1, and the other positions are assigned the value 0. When the type is a target-related interaction state type, the system extracts the relative motion state of the associated target traffic participants and calculates the target state feature vector. The target state feature vector contains the following three physical variables with unified dimensions: (1) Relative distance (dimension in meters): The relative distance is equal to the square of the difference between the horizontal coordinates of the vehicle node and the associated target node, plus the square of the difference between the vertical coordinates, and then the sum of the two is taken as the square root; (2) Relative speed (dimension in meters per second): The relative speed is equal to the projection of the velocity vector of the vehicle node onto the unit direction vector of the line connecting the two nodes, minus the projection of the velocity vector of the associated target node onto the unit direction vector; (3) Relative acceleration (dimension in meters per square second): The relative acceleration is equal to the projection of the acceleration vector of the vehicle node onto the unit direction vector of the line connecting the two nodes, minus the projection of the acceleration vector of the associated target node onto the unit direction vector. The above three values ​​constitute a vector with dimension 3, which serves as the extracted dynamic interaction feature. To meet the requirement of fixed input tensor dimensions for neural networks, the system performs dimension alignment before feature concatenation: when the dynamic interaction feature is a 4-dimensional one-hot encoded vector of discrete interaction event type, it remains unchanged; when it is a 3-dimensional vector of target-related interaction state type, zero-value padding is performed at the end to expand it into a unified 4-dimensional vector.

[0057] The system extracts a fixed-length sequence of historical motion states for each traffic participant. The length of the observation time window is usually set to the sum of the time for human drivers or intelligent systems in the three stages of perception delay, decision-making time, and execution response.

[0058] Using the region of interest alignment operator, the system simultaneously extracts a local spatial feature map from the fused feature matrix of the dynamic traffic situation map, centered on the real-time coordinates of traffic participants. The conventional calculation logic for the grid side length of this feature map is as follows: first, take the average length of a typical vehicle plus twice the preset safety redundancy distance, then divide the sum by the physical resolution of the grid, and finally round the calculated result up.

[0059] Subsequently, the processed temporal features are fed into an attention-based prediction model (such as the Wayformer derivative network used in this example). The model consists of two parts: an encoder and a decoder. The encoder contains 6 self-attention layers and the hidden layer dimension is set to 256. The decoder uses the intent query vector and the encoder output to perform cross-attention calculation, thereby deduce a series of future position coordinates and generate the final predicted trajectory.

[0060] The aforementioned time-series prediction model derived from Wayformer underwent basic training in the cloud before being deployed to edge devices. Basic training utilized a real traffic trajectory dataset consistent with the actual application scenario of this embodiment, using the historical state sequences and corresponding dynamic interaction features of each traffic participant as encoder input, and the actual future location coordinate sequences as supervision labels. The loss function structure used in the basic training phase remained completely consistent with the composite error function (i.e., the weighted sum of the average displacement error minADE, the final displacement error minFDE, and the regularization penalty term) described in the subsequent cloud-based offline optimization module. Through backpropagation iteration using massive historical normal and hazardous operating condition data, the time-series prediction model acquired basic trajectory extrapolation capabilities, serving as the baseline model for subsequent cloud-based adaptive optimization.

[0061] To assess the probability of spatiotemporal overlap between the predicted trajectories of different traffic participants, the system introduces two evaluation metrics measured in seconds: the first is collision time, calculated by subtracting the average length of a typical vehicle from the relative distance between the two nodes, and then dividing by the relative approach speed of both parties. If the two parties do not approach each other (i.e., the relative speed is less than or equal to zero), this metric is considered positive infinity. The second is intrusion time, which is equal to the absolute value of the time difference between the moment when the previous vehicle completely leaves the potential conflict area and the moment when the other vehicle just enters the same area.

[0062] The system calculates the minimum collision time and minimum intrusion time within the predicted field of vision for each pair of traffic participants. To unify time indicators with different dimensions, the system outputs a comprehensive potential traffic conflict risk quantification value between 0 and 1. The calculation formula is as follows: the comprehensive potential traffic conflict risk quantification value equals the preset first weighting coefficient multiplied by (the reciprocal of the sum of the minimum collision time and the minimum normal number), plus the preset second weighting coefficient multiplied by the reciprocal of the sum of the minimum intrusion time and the minimum normal number, and finally the sum of the two is input into the Sigmoid function for calculation.

[0063] The smallest positive constant is set to 10. -5 To prevent the denominator from being zero, the sum of the first weighting coefficient and the second weighting coefficient is always equal to 1; the first weighting coefficient is equal to the average speed of the current traffic scenario divided by the maximum speed limit of the road, and the quotient is multiplied by the base coefficient. The base coefficient ranges from 0.5 to 1, so that the higher the speed, the more emphasis is placed on the minimum collision time index to deal with the risk of emergency direct collision; the second weighting coefficient is equal to 1 minus the first weighting coefficient.

[0064] By fusing dynamic interaction features extracted from structured interaction relationships with the historical motion state sequences of traffic participants as input to a time-series prediction model, the model fully considers the complex influence of the surrounding environment and the behavior of other participants on their future decisions. This allows the model to predict driving trajectories in highly interactive scenarios such as intersections and ramp merging.

[0065] Furthermore, by monitoring the traffic prediction data, when the potential traffic conflict risk exceeds a preset risk threshold, the dynamic traffic situation map, structured interaction relationships, and traffic prediction data are encapsulated into critical traffic event data; the specific implementation process includes: The potential traffic conflict risks in the traffic prediction data are monitored in a streaming manner, and the quantified value of the potential traffic conflict risk is continuously compared with a preset risk threshold. When the quantified value exceeds the risk threshold, an event recording program is triggered to capture the dynamic traffic situation map, the structured interaction relationship, the traffic prediction data at the trigger time, and the traffic evolution data at the trigger time within a preset time window before the trigger time. A unique event identifier and timestamp are assigned to the captured data set, and the data is integrated into structured critical traffic event data. The critical traffic event data is encrypted using an asymmetric encryption algorithm, and the encrypted data is uploaded to the cloud server through a secure transmission protocol.

[0066] Specifically, the process for generating critical traffic event data is as follows: The system performs streaming monitoring of the quantified potential traffic conflict risk values ​​in the generated traffic prediction data and continuously compares them with preset risk thresholds. The preset risk thresholds are calculated by substituting the collision time into the safety model and are used to balance the sensitivity and false alarm rate of the system's early warning at ordinary intersections in urban areas.

[0067] When a quantized value exceeds a threshold, the edge computing device triggers an event logging program. This program backtracks to the circular buffer, capturing data from a preset time window before the trigger and traffic evolution data from a preset time window after the trigger. The common method for selecting the preset time window length is: the preceding time window length is greater than or equal to the quotient obtained by dividing the vehicle's maximum permissible speed by its maximum braking deceleration; the following time window length is set based on the typical energy dissipation after a traffic accident collision and the time required for the vehicle to come to a complete stop.

[0068] The system uses a built-in hardware random number generator to generate a unique event identifier. At the same time, it calls the main system clock of the edge computing device (which has been synchronized with the clocks of various traffic safety facility sensors through a precise time protocol) to read the actual physical moment of the trigger as a timestamp. The identifier and the actual timestamp are then integrated into structured critical traffic event data.

[0069] To ensure secure data transmission, the edge roadside unit employs an asymmetric encryption algorithm. To balance data processing speed, the system first uses Advanced Encryption Standard (AES) with a 256-bit key length to symmetrically encrypt traffic event data; then, it uses an asymmetric encryption algorithm based on elliptic curve cryptography to encrypt the aforementioned symmetric key. A secure transmission protocol is established to upload the encrypted data to the cloud server.

[0070] By monitoring potential traffic conflict risks in real time and triggering a recording procedure when the risk exceeds a dynamic threshold, the most valuable difficult samples or edge cases for model optimization can be captured from massive traffic flows, integrated into structured event data packets, and uploaded in encrypted form, which is beneficial to the security of data during transmission and storage.

[0071] Furthermore, the time-series prediction model is adaptively optimized using the aforementioned critical traffic event data; see [reference needed]. Figure 3 The specific implementation process includes: The received critical traffic event data is divided into a training set and a validation set. Data augmentation is performed on the training set by perturbing the initial state of traffic participants to generate diverse derivative risk scenarios. Original traffic prediction data and traffic evolution data are extracted from each data point in the training set, and the prediction error is calculated as a loss function. Using the loss function, the network weights of the time-series prediction model are fine-tuned and updated through the backpropagation algorithm. The performance of the updated time-series prediction model is evaluated using the validation set.

[0072] Specifically, the process of adaptive optimization is as follows: After the data is decrypted on the cloud server, the dataset is divided into a training set and a validation set (typically in an 80% to 20% ratio). The system performs data augmentation on the training set by applying statistical perturbations to the initial state vectors of traffic participants. Derivative risk scenarios are generated by incorporating random speed noise that follows a zero-mean Gaussian distribution. The standard deviation of this random speed noise is specifically chosen such that its value equals the upper limit of the absolute value of the limiting speed measurement error of the worst-performing sensor in the multimodal fusion module, to simulate extreme perception conditions.

[0073] During training, the prediction error is calculated as the loss function. The composite error function consists of the average displacement error and the final displacement error (both measured in meters): ; Explanation of the meaning of variables in the formula: The total number of steps to predict the field of view; K represents the total number of possible trajectory modes output by the decoder; k represents a specific mode among the multiple possible trajectories output by the decoder; Let be the predicted horizontal and vertical coordinates at time t from the model output, in meters. Let be the actual objective coordinates occurring at time t, with the dimension of meters; This is a regularization penalty term for the network weights of the model, specifically represented as the sum of squares of all network weights in the model (L2 norm), used to prevent the model from overfitting on small sample risk data; These are the weighting coefficients. The constant value is 1.0; because the collision avoidance system requires higher accuracy at the end point of the trajectory than during the intermediate process, Must meet The specific selection formula is as follows: ,in A preset endpoint focus enhancement constant is defined, the value of which is positively correlated with the vehicle's current average speed (0.5 under normal urban speed limit conditions). The system's preset regularization constant is divided by the total number of training set data samples.

[0074] The system applies the backpropagation algorithm and uses an optimizer with an initial learning rate of 0.00001 to fine-tune the network weights, outputting an optimized time series prediction model.

[0075] After each training cycle, the model is evaluated using the validation set. The early stopping mechanism is set when the validation set loss does not decrease for five consecutive iterations.

[0076] By employing data augmentation techniques, the number of samples in risk scenarios was expanded. The error between the predicted trajectory and the actual trajectory was calculated as the loss function, and a fine-tuning update strategy was adopted. This enabled the model to specifically correct its erroneous perceptions in dangerous scenarios while retaining its original knowledge.

[0077] Furthermore, test scenarios are generated through a simulation platform to conduct stress tests and security verifications, resulting in an update package; the specific implementation process includes: The critical traffic event data is reconstructed in the simulation platform, and a series of edge test scenarios are generated by perturbing the key parameters in the scenario. The optimized time series prediction model and the unoptimized time series prediction model are compared and tested in the edge test scenarios to evaluate key safety indicators, including prediction accuracy, false negative rate, and false alarm rate. When the key safety indicators of the optimized time series prediction model are better than the preset verification standard, they are packaged into an encrypted update package.

[0078] Specifically, the process for conducting stress testing and safety verification is as follows: The system reconstructs the road topology and target trajectory in a high-fidelity autonomous driving simulation platform using an open standard interface. It then uses Monte Carlo sampling to perturb key parameters and generate edge testing scenarios. For example, the road surface friction coefficient is perturbed from the standard 0.8 to a value between 0.15 and 0.35 to simulate the physical scenario of braking distance changes caused by rain and snow. The ambient lighting parameters are adjusted in the simulation engine from midday values ​​to a backlit morning state.

[0079] For the above scenarios, the model needs to be compared and tested on a simulation platform before and after optimization, focusing on three key safety indicators: The first is prediction accuracy, which mainly measures the proportion of samples where the longitudinal and lateral errors between the predicted trajectory and the actual trajectory do not exceed the limits within a specified time. Here, the upper limit of the longitudinal error is the safety margin of the braking distance at the same vehicle speed, and the upper limit of the lateral error is half the difference between the standard lane width and the conventional vehicle width. The passing score for this proportion must directly align with the minimum requirements of relevant autonomous driving safety standards. The second is the false alarm rate, which is the proportion of cases where a collision occurs in the simulation but no warning is triggered. Its threshold must not exceed the maximum accident red line designed by the system. The third is the false alarm rate, which refers to the proportion of alarms that are falsely triggered when no danger has occurred. To avoid driver alarm fatigue, this indicator is strictly limited to the highest intervention frequency allowed by the system.

[0080] The deployment process will only be triggered when the optimized model outperforms the established standards in all metrics. In practice, the system will first use a hash algorithm to generate a checksum for the network configuration file and the fine-tuned weight file, and then call the public key infrastructure to encapsulate it into an encrypted update package. Edge devices will download, decrypt and replace the package during network downtime.

[0081] The rigorous update and deployment mechanism aims to prevent performance degradation or the introduction of new security risks during model iterations.

[0082] This invention provides a highly integrated fusion perception system based on road traffic safety facilities. Leveraging multimodal deep fusion and interactive cognitive technologies, the system's capabilities have evolved from simply identifying isolated targets to quantitatively analyzing the dynamic game between traffic participants. Combined with forward-looking prediction, the previously passive and lagging response mechanism is transformed into proactive risk foresight. Furthermore, relying on a collaborative closed loop of "problem discovery at the edge—optimization in the cloud—deployment at the edge," roadside infrastructure truly possesses the ability to self-learn and evolve, continuously improving its assessment of real-world road network risks.

[0083] Example 2 This embodiment provides a detailed explanation of the process of extracting intermediate layer features for each modality, extracting the relative motion state of the target, and calculating the weighted sum of prediction errors based on the intermediate layer feature vector, target state feature vector, and composite error function generated in Embodiment 1.

[0084] The system performs spatiotemporal synchronization and deep fusion of heterogeneous raw data to generate a dynamic traffic situation map. A spatiotemporal interaction relationship graph is constructed from the dynamic traffic situation map, and inference is performed using an inference model to output structured interaction relationships. Dynamic interaction features are extracted from the structured interaction relationships, and the dynamic traffic situation map is fused with a time-series prediction model to generate traffic prediction data. The traffic prediction data is monitored, and when the potential traffic conflict risk exceeds a preset risk threshold, it is encapsulated as critical traffic event data and encrypted before being uploaded. The critical traffic event data is used to adaptively optimize the time-series prediction model. Test scenarios are generated through a simulation platform for stress testing and security verification to obtain an update package.

[0085] Furthermore, after extracting the intermediate layer feature vectors, the system traverses the feature grid of the bird's-eye view and compares the mean difference of the channel activations of heterogeneous sensors. If the spatial offset exceeds the normal tolerance range introduced by time alignment, a mask is used to filter the corresponding visual features to eliminate asynchronous artifacts.

[0086] Specifically, after acquiring the intermediate layer feature vectors of the multimodal systems, to further eliminate the spatial misalignment artifacts caused by differences in the sampling of the underlying hardware, the system performs strict feature consistency alignment before feature tensor concatenation. The system traverses the preset bird's-eye view feature grid layer by layer, independently extracting the channel values ​​corresponding to the point cloud and radar intermediate layer feature vectors at each grid location, and calculating their mean as the local activation level of that region. Combining the tolerable time difference value established in the time synchronization step, the system directly reuses its corresponding derived physical spatial drift as a standard for fault tolerance. Since each grid has a predetermined physical size, the system performs comparative calibration using the local neighborhood range formed by the current grid and its adjacent grids. If the activation difference between the point cloud and the radar within this neighborhood does not exceed half of their mean values, the system determines it as valid physical feedback; if there is no match, it is determined that a severe asynchronous time difference has occurred in that local area. Because camera data has the highest frame rate and shortest exposure time, it is most sensitive to temporal deviations. If the point cloud and radar (with a longer sampling period) cannot achieve spatial consistency matching within a grid area, it indicates that the multimodal timestamp alignment at that location within the current time window has failed, easily leading to sensor ghosting artifacts. To prevent such severe asynchronous artifacts from polluting subsequent high-order fusion decisions, the system uses the activation difference between the point cloud and radar as a proxy detection indicator of the temporal misalignment degree in that area. Once a failure is detected, the system generates an equal-dimensional all-zero mask and performs element-wise multiplication with the image intermediate layer feature vector at the corresponding grid location, thereby masking unreliable visual features in that area. This process effectively suppresses spatial ghosting caused by asynchronous sensor sampling.

[0087] Furthermore, after extracting the target state feature vector, the ratio of the relative distance to the known dynamic sensing radius threshold is extracted. The risk approximation coefficient is derived by subtracting this ratio from the unit constant, and then scaled element by element with the feature vector to highlight high-risk interaction features.

[0088] Specifically, after calculating the relative motion parameters and concatenating them into a target state feature vector, the system internally performs an adaptive weighting process based on dynamic boundaries to highlight the characteristics of high-risk interaction objects before inputting them into the prediction network. The system directly retrieves the dynamic perception radius threshold calculated during the earlier graph node construction phase. This threshold is the larger of the basic focus range constant and the extended boundary derived from the absolute velocity. The system divides the actual relative distance in the currently extracted feature vector by this dynamic perception radius threshold. Subsequently, it subtracts the aforementioned ratio from the unit constant to derive a risk approximation coefficient distributed within the normalized interval. The physical logic of this coefficient is: the closer the target is to the perception boundary, the closer the coefficient tends to the underlying minimum; the closer it is to the vehicle itself, the closer the coefficient tends to the full-load maximum. Next, the system uses this approximation coefficient as a global scalar and performs element-wise multiplication with the relative state values ​​in the target state feature vector, outputting the scaled target state feature vector. This process adaptively amplifies the feature weights of close-range high-risk targets, guiding the time-series model to focus on the emergency rendezvous state.

[0089] Furthermore, when calculating the composite error function, the absolute value of the random velocity noise injected during the data augmentation stage is extracted, divided by the existing velocity measurement error standard deviation parameter, and the resulting physical ratio is used to dynamically amplify the weighting coefficient of the final displacement error, thereby strengthening the penalty mechanism.

[0090] Specifically, during the training process, when extracting augmented data and calculating the composite error function, the system performs dynamic loss weight allocation on the perturbed training set to improve the model's fitting ability to dangerous edge scenarios. When dealing with derived risk scenarios, the system internally traces and extracts the absolute value of the random velocity noise applied to the participants' initial state during the data augmentation phase. This absolute value is then divided by the system's preset Gaussian distribution velocity measurement error standard deviation parameter to calculate the basic physical ratio. Before calculating the loss function, the system dynamically multiplies this ratio with the system's preset final displacement error weight coefficient. The generated new coefficient replaces the original displacement weight parameter in the summation of the composite error function, while the coefficients of the remaining average displacement error and regularization penalty term remain unchanged. The logic of this mechanism is that the more severe the perturbation injected during augmentation, the more extreme the simulated perception error; the system then significantly increases the penalty constraint for endpoint prediction by reusing this ratio. This process enables the model to adaptively apply a larger endpoint error penalty when dealing with highly perturbated samples, enhancing the network's fine-tuning specificity.

[0091] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A highly integrated fusion perception system based on road traffic safety facilities, characterized in that, include: Multimodal fusion module: performs spatiotemporal synchronization and deep fusion of heterogeneous raw data from traffic safety facilities to generate dynamic traffic situation maps; Interaction Relationship Recognition Module: Constructs a spatiotemporal interaction relationship diagram of the traffic scenario through the dynamic traffic situation map, and infers the spatiotemporal interaction relationship diagram through the reasoning model to output structured interaction relationships; Prediction and monitoring module: Extracts dynamic interaction features from the structured interaction relationship, fuses the dynamic traffic situation map through a time-series prediction model, and generates traffic prediction data containing driving trajectories and potential traffic conflict risks; monitors the traffic prediction data, and when the potential traffic conflict risk exceeds a preset risk threshold, encapsulates the dynamic traffic situation map, structured interaction relationship, and traffic prediction data into critical traffic event data, and uploads it in encryption. Cloud-based offline optimization module: Adaptively optimizes the time-series prediction model using the critical traffic event data; generates test scenarios through a simulation platform, conducts stress tests and safety verifications, and obtains an update package.

2. The highly integrated fusion perception system based on road traffic safety facilities according to claim 1, characterized in that, The specific process of the multimodal fusion module includes: performing time alignment based on a unified timestamp on heterogeneous raw data from traffic safety facilities including cameras, lidar, and millimeter-wave radar, and achieving spatial alignment through coordinate system transformation; extracting intermediate layer feature vectors from the aligned image data, point cloud data, and radar data using a deep learning network; weighting and fusing the intermediate layer feature vectors to generate a feature representation under a unified bird's-eye view, and generating the dynamic traffic situation map based on the feature representation.

3. The highly integrated fusion perception system based on road traffic safety facilities according to claim 1, characterized in that, The specific process of the interaction relationship cognition module includes: identifying traffic participants from the dynamic traffic situation map, treating each traffic participant as a graph node; establishing graph edges between graph nodes with potential interactive behaviors based on the spatial proximity and motion vector relationships between the traffic participants, jointly constructing the spatiotemporal interaction relationship graph; extracting the dynamic attributes of each graph node, including position, speed, acceleration, and heading angle, as node features; inputting the spatiotemporal interaction relationship graph with node features into the inference model, updating the node features through the message passing and aggregation mechanism of the inference model to integrate the interactive influence of neighboring nodes; decoding the updated node features, outputting the structured interaction relationship, which includes quantified vehicle following, lane changing intentions, and pedestrian-vehicle conflict risks.

4. The highly integrated fusion perception system based on road traffic safety facilities according to claim 1, characterized in that, The specific process for generating the traffic prediction data includes: extracting historical motion state sequences for each traffic participant in the dynamic traffic situation map, and extracting dynamic interaction features of quantified interaction relationships from the structured interaction relationships; concatenating the historical motion state sequences with the dynamic interaction features, and integrating the spatial features of the dynamic traffic situation map to construct a temporal input feature vector for each traffic participant; inputting the temporal input feature vector into the temporal prediction model, which outputs a sequence of position coordinates of the traffic participants at multiple future times to form a predicted driving trajectory; assessing the potential traffic conflict risk by calculating the spatiotemporal overlap probability of the predicted driving trajectories of different traffic participants; and merging the predicted driving trajectories of all traffic participants with the potential traffic conflict risks to generate the traffic prediction data.

5. A highly integrated fusion perception system based on road traffic safety facilities according to claim 4, characterized in that, The specific process of extracting dynamic interaction features from the structured interaction relationship includes: identifying the specific type of the structured interaction relationship, wherein the type includes a preset discrete interaction event type and a target-associated interaction state type; when the type is a discrete interaction event type, generating a one-hot encoded vector representing the event classification as the dynamic interaction feature; when the type is a target-associated interaction state type, extracting the relative distance, relative speed, and relative acceleration of the associated target traffic participants to form a target state feature vector as the dynamic interaction feature.

6. A highly integrated fusion perception system based on road traffic safety facilities according to claim 4, characterized in that, The specific generation process of the critical traffic event data includes: streaming monitoring of potential traffic conflict risks in the traffic prediction data, continuously comparing the quantified value of the potential traffic conflict risk with a preset risk threshold; when the quantified value exceeds the risk threshold, triggering an event recording program to capture the dynamic traffic situation map, the structured interaction relationship, the traffic prediction data at the trigger time, and the traffic evolution data within a preset time window after the trigger time within a preset time window; assigning a unique event identifier and timestamp to the captured data set, integrating it into structured critical traffic event data; encrypting the critical traffic event data using an asymmetric encryption algorithm, and uploading the encrypted data to a cloud server through a secure transmission protocol.

7. A highly integrated fusion perception system based on road traffic safety facilities according to claim 1, characterized in that, The specific process of adaptive optimization includes: dividing the received critical traffic event data into a training set and a validation set; performing data augmentation on the training set by perturbing the initial state of traffic participants to generate diverse derivative risk scenarios; extracting original traffic prediction data and traffic evolution data from each data point in the training set, calculating the prediction error as a loss function; using the loss function, fine-tuning and updating the network weights of the time-series prediction model through a backpropagation algorithm to output the optimized time-series prediction model; and using the validation set to evaluate the performance of the optimized time-series prediction model.

8. The highly integrated fusion perception system based on road traffic safety facilities according to claim 1, characterized in that, The specific process of stress testing and safety verification includes: reconstructing the critical traffic event data in the simulation platform, and generating a series of edge test scenarios by perturbing the key parameters in the scenario; comparing the optimized time series prediction model with the unoptimized time series prediction model in the edge test scenarios to evaluate key safety indicators, including prediction accuracy, false negative rate, and false alarm rate; when the key safety indicators of the optimized time series prediction model are better than the preset verification standard, it is packaged into an encrypted update package.

Images