Simulation evaluation method and device for car-road cooperation automatic driving model

Abstract

The application provides a simulation evaluation method and device for a vehicle-road cooperation automatic driving model, and relates to the technical field of automatic driving testing. The method comprises the following steps: acquiring a multi-source data packet of a real traffic scene, and configuring a simulation platform based on the multi-source data packet; in the case that a to-be-tested automatic driving model accesses the configured simulation platform, injecting vehicle-road cooperation perception information into the automatic driving model to obtain driving behavior instructions; based on a preset real traffic participant behavior prior model, performing behavior reasoning processing on a driving behavior currently performed by a simulation vehicle and a real-time traffic state fed back by the simulation platform to obtain motion response instructions of a plurality of traffic participants, and performing multi-dimensional evaluation processing on driving performance of the automatic driving model based on a plurality of evaluation dimensions to obtain an evaluation result of the automatic driving model. The application improves the consistency between the simulation evaluation result of the automatic driving model and the real road driving performance.

Description

Technical Field

[0001] This invention relates to the field of autonomous driving testing technology, and in particular to a simulation evaluation method and apparatus for vehicle-road cooperative autonomous driving models. Background Technology

[0002] Autonomous driving simulation evaluation is a key technical means to verify the performance and safety of autonomous driving algorithms. Existing autonomous driving simulation evaluation methods mainly rely on artificially constructed virtual traffic scenarios or rule-based traffic flows. By running the algorithm under test in a simulation platform and recording driving data, the perception, decision-making, and control capabilities of the algorithm are quantitatively evaluated.

[0003] However, the aforementioned existing methods have significant technical limitations. First, since both the virtual scene and traffic flow are constructed manually or through rules, the behavioral patterns of traffic participants and the dynamic characteristics of the scene systematically deviate from the real urban traffic environment. This makes it difficult for simulation evaluation results to accurately reflect the actual performance of autonomous driving algorithms in real traffic scenarios. Second, existing simulation evaluation methods are mainly designed for single-vehicle intelligent architectures. The simulation environment only provides perception information from the perspective of onboard sensors and lacks a mechanism for simulating and injecting global traffic state information acquired by roadside perception devices. Therefore, it cannot support the evaluation needs of autonomous driving models with vehicle-road cooperative capabilities. Furthermore, the motion behaviors of other traffic participants in existing methods are usually generated using preset rules or simple replay methods. They cannot provide dynamic and reasonable interactive responses based on the actual driving behavior of the autonomous driving model under test. This results in a significant difference between the traffic flow evolution in the simulation process and the human-machine game behavior in real traffic, limiting the effectiveness of the evaluation results in measuring the long-term decision-making ability of the algorithm.

[0004] Therefore, how to evaluate autonomous driving models is a technical problem that urgently needs to be solved. Summary of the Invention

[0005] This invention provides a simulation evaluation method and apparatus for vehicle-road cooperative autonomous driving models. By driving the configuration of the simulation platform with real traffic scene data, injecting multi-source perception information, and dynamic interactive reasoning based on behavioral prior models, the traffic evolution in the simulation environment is highly consistent with real traffic, thereby improving the consistency between the simulation evaluation results of autonomous driving models and real road driving performance.

[0006] In a first aspect, the present invention provides a simulation evaluation method for vehicle-road cooperative autonomous driving models, comprising the following steps: Acquire multi-source data packets from real traffic scenarios and configure a simulation platform based on the multi-source data packets; When the autonomous driving model to be tested is connected to the configured simulation platform, vehicle-road cooperative perception information is injected into the autonomous driving model to obtain driving behavior instructions; wherein, the driving behavior instructions are used to drive the movement of the simulated vehicle in the simulation platform, the simulated vehicle is configured with the autonomous driving model, and the vehicle-road cooperative perception information is obtained based on the multi-source data packets; Based on a pre-defined prior model of real traffic participant behavior, behavioral reasoning is performed on the driving behavior currently executed by the simulated vehicle and the real-time traffic status fed back by the simulation platform to obtain motion response commands for multiple traffic participants, and the motion status of the multiple traffic participants is updated according to the motion response commands. Under the condition that the simulated vehicle interacts with the multiple traffic participants for a preset duration, the driving performance of the autonomous driving model is evaluated in a multi-dimensional manner based on multiple evaluation dimensions to obtain the evaluation result of the autonomous driving model.

[0007] Secondly, the present invention also provides a simulation evaluation device for vehicle-road cooperative autonomous driving models, comprising the following modules: The acquisition module is used to acquire multi-source data packets from real traffic scenarios and configure the simulation platform based on the multi-source data packets; The injection module is used to inject vehicle-road cooperative perception information into the autonomous driving model under test when the autonomous driving model is connected to the configured simulation platform to obtain driving behavior instructions; wherein, the driving behavior instructions are used to drive the movement of the simulated vehicle in the simulation platform, the simulated vehicle is configured with the autonomous driving model, and the vehicle-road cooperative perception information is obtained based on the multi-source data packet; The reasoning module is used to perform behavioral reasoning processing on the driving behavior currently performed by the simulated vehicle and the real-time traffic status fed back by the simulation platform based on a preset prior model of the behavior of real traffic participants, to obtain motion response commands of multiple traffic participants, and to update the motion status of the multiple traffic participants according to the motion response commands. The evaluation module is used to perform multi-dimensional evaluation of the driving performance of the autonomous driving model based on multiple evaluation dimensions, under the condition that the simulated vehicle interacts with the multiple traffic participants for a preset time, and obtain the evaluation result of the autonomous driving model.

[0008] Thirdly, the present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the simulation evaluation method for a vehicle-road cooperative autonomous driving model as described above.

[0009] Fourthly, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the simulation evaluation method for vehicle-road cooperative autonomous driving models as described above.

[0010] Fifthly, the present invention also provides a computer program product, including a computer program that, when executed by a processor, implements the simulation evaluation method for a vehicle-road cooperative autonomous driving model as described above.

[0011] This invention provides a simulation evaluation method and apparatus for vehicle-road cooperative autonomous driving models. The method involves acquiring multi-source data packets from real traffic scenarios and configuring a simulation platform based on these packets. When the autonomous driving model to be tested is connected to the configured simulation platform, vehicle-road cooperative perception information is injected into the model to obtain driving behavior commands. These commands drive the movement of a simulated vehicle within the simulation platform. The simulated vehicle is configured with the autonomous driving model, and the vehicle-road cooperative perception information is obtained based on the multi-source data packets. Based on a preset prior model of real traffic participant behavior, behavioral reasoning is performed on the current driving behavior of the simulated vehicle and the real-time traffic status fed back by the simulation platform to obtain motion response commands for multiple traffic participants. The motion status of these multiple traffic participants is then updated according to these motion response commands. After the simulated vehicle interacts with the multiple traffic participants for a preset duration, the driving performance of the autonomous driving model is evaluated using multiple evaluation dimensions to obtain the evaluation result of the autonomous driving model. By using real traffic scenario data to drive the configuration of the simulation platform, injecting multi-source perception information, and using dynamic interactive reasoning based on behavioral prior models, the traffic evolution in the simulation environment is highly consistent with real traffic, thereby improving the consistency between the simulation evaluation results of the autonomous driving model and the driving performance on real roads. Attached Figure Description

[0012] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0013] Figure 1 This is one of the flowcharts of the simulation evaluation method for vehicle-road cooperative autonomous driving models provided by the present invention.

[0014] Figure 2 This is the second flowchart of the simulation evaluation method for vehicle-road cooperative autonomous driving models provided by this invention.

[0015] Figure 3 This is a schematic diagram of the structure of the simulation and evaluation device for vehicle-road cooperative autonomous driving model provided by the present invention.

[0016] Figure 4 This is a schematic diagram of the structure of the electronic device provided by the present invention. Detailed Implementation

[0017] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0018] All actions involving the acquisition of signal information or data in this invention are carried out in compliance with the relevant data protection laws and policies of the country where the device is located, and with the authorization granted by the owner of the device.

[0019] The following is combined Figures 1-4 This invention describes a simulation evaluation method and apparatus for vehicle-road cooperative autonomous driving models. By driving the configuration of the simulation platform with real traffic scenario data, injecting multi-source perception information, and using dynamic interactive reasoning based on behavioral prior models, the traffic evolution in the simulation environment is highly consistent with real traffic, thereby improving the consistency between the simulation evaluation results of the autonomous driving model and the driving performance on real roads.

[0020] Figure 1 This is one of the flowcharts illustrating a simulation evaluation method for a vehicle-road cooperative autonomous driving model provided by the present invention, such as... Figure 1 As shown, the method may include, but is not limited to, steps S100 to S400: S100: Acquire multi-source data packets from real traffic scenarios and configure the simulation platform based on the multi-source data packets; S200, when the autonomous driving model to be tested is connected to the configured simulation platform, vehicle-road cooperative perception information is injected into the autonomous driving model to obtain driving behavior instructions; wherein, the driving behavior instructions are used to drive the movement of the simulated vehicle in the simulation platform, the simulated vehicle is configured with the autonomous driving model, and the vehicle-road cooperative perception information is obtained based on the multi-source data packet; S300, based on a preset prior model of real traffic participant behavior, performs behavioral reasoning processing on the driving behavior currently executed by the simulated vehicle and the real-time traffic status fed back by the simulation platform to obtain motion response commands for multiple traffic participants, and updates the motion status of the multiple traffic participants according to the motion response commands. S400, under the condition that the simulated vehicle continuously interacts with the multiple traffic participants for a preset time, the driving performance of the autonomous driving model is evaluated in a multi-dimensional manner based on multiple evaluation dimensions to obtain the evaluation result of the autonomous driving model.

[0021] refer to Figure 1 and Figure 2 As shown, in step S100 of some embodiments, multi-source data packets of real traffic scenarios are obtained, and a simulation platform is configured based on the multi-source data packets.

[0022] It is understandable that multi-source data packets refer to a comprehensive dataset containing various types of data, collected and processed from real traffic environments. This dataset at least covers road structure data, traffic participant trajectory data, traffic signal status data, vehicle sensor data, and roadside perception data.

[0023] Road structure data refers to information describing the static physical structure of a road, such as its geometry, number and width of lanes, intersection connections, and the location of traffic signs and markings. Traffic participant trajectory data refers to time-series data recording the continuous positional changes of various traffic participants, including motor vehicles, non-motorized vehicles, and pedestrians, over a period of time.

[0024] Traffic signal status data refers to information recording the phase of traffic lights at different times, including green, yellow, and red light phases. Vehicle-mounted sensor data refers to raw perception data collected by sensors such as cameras and lidar installed on actual vehicles. Roadside perception data refers to raw perception data collected by roadside sensor units installed on poles or structures at intersections.

[0025] Real-world traffic scenarios can be selected from road areas with complex interactive characteristics, such as typical urban intersections, expressway ramp entrances and exits, and roundabouts. During the data collection process, a high-precision integrated navigation and positioning system, multiple types of cameras, and multi-beam LiDAR are installed on the data collection vehicle, while fixed-view cameras and LiDAR are installed on roadside poles. All data collection devices use the standard time provided by the Global Navigation Satellite System as a unified time source for data labeling.

[0026] A simulation platform refers to a computer simulation software environment capable of vehicle dynamics simulation, sensor simulation, and traffic flow simulation. This embodiment uses the CARLA simulator as a typical simulation platform. This simulator provides an open-source autonomous driving simulation environment that supports custom map loading, multi-sensor simulation, and vehicle physics model solving. Other available simulation platforms include a co-simulation environment combining the SUMO traffic simulator and CARLA, or commercial simulation platforms based on VTD or LGSVL.

[0027] The specific operations for configuring the simulation platform include constructing a digital twin static scene based on road structure data from multi-source data packets, generating dynamic traffic flow based on traffic participant trajectory data, setting the control timing of virtual traffic lights based on traffic signal status data, and configuring virtual sensor parameters based on vehicle-mounted sensor data and roadside perception data. A digital twin static scene refers to a virtual scene model that is highly consistent with the real scene in terms of geometric dimensions and appearance features, obtained through three-dimensional digital reconstruction of a real road scene. Dynamic traffic flow refers to a traffic flow simulation environment running in the simulation platform, composed of multiple traffic participants with independent movement behaviors.

[0028] In step S200 of some embodiments, when the autonomous driving model to be tested is connected to the configured simulation platform, vehicle-road cooperative perception information is injected into the autonomous driving model to obtain driving behavior instructions; wherein, the driving behavior instructions are used to drive the movement of the simulated vehicle in the simulation platform, the simulated vehicle is configured with the autonomous driving model, and the vehicle-road cooperative perception information is obtained based on the multi-source data packet.

[0029] Understandably, the autonomous driving model to be tested refers to an autonomous driving algorithm model with an end-to-end or modular architecture, which can directly or indirectly output vehicle control commands based on the input perception information.

[0030] It should be noted that the autonomous driving model under test refers to the implementation of the autonomous driving algorithm being evaluated. This model can adopt an end-to-end neural network architecture, that is, a deep learning model that directly maps raw sensor data to vehicle control commands; or it can adopt a modular architecture, that is, an algorithm system that includes perception, prediction, planning, and control modules respectively. An end-to-end neural network architecture means that a single deep neural network is used to directly complete the mapping from sensor input to control command output, without explicit decomposition of intermediate tasks such as target detection and trajectory prediction.

[0031] There are two ways to connect autonomous driving models to the simulation platform: one is to compile the model into a client program compatible with the simulation platform and interact with the simulation environment through the application programming interface provided by the simulation platform; the other is to deploy the model as an independent process or container and exchange data with the simulation platform through network communication middleware.

[0032] Furthermore, vehicle-to-infrastructure (V2I) cooperative perception information refers to fused perception information that simultaneously includes local perception data from the perspective of onboard sensors and global perception data from the perspective of roadside sensors. Local perception data from the perspective of onboard sensors refers to first-person perception data centered on the vehicle, obtained from virtual sensors mounted on the simulated vehicle; its field of view is limited by the vehicle's position and the sensor's mounting angle. Global perception data from the perspective of roadside sensors refers to perception data from virtual sensors mounted on roadside poles, viewed from a bird's-eye view or a global intersection perspective; it can cover blind spots that onboard sensors cannot observe due to obstruction or field-of-view limitations. Injection refers to the real-time, continuous transmission of the aforementioned V2I cooperative perception information to the perception input interface of the autonomous driving model in the form of structured messages through the simulation's communication middleware, enabling the tested model to obtain multi-view perception capabilities equivalent to a real V2I cooperative system in the simulation environment. The communication middleware refers to the intermediate layer software component responsible for data transmission between different software modules, and can be implemented using a solution based on a robot operating system or a custom message bus. Structured messages are message units organized according to a predefined data format and containing timestamps, data type identifiers, sensor parameters, and sensed data volumes.

[0033] Driving behavior commands refer to the instruction signals obtained from the output port of the autonomous driving model, used to control the motion of the simulated vehicle. These command signals may include target steering wheel angle, target throttle opening, target brake master cylinder pressure, or target longitudinal acceleration requests. If the autonomous driving model outputs a trajectory point sequence, then the driving behavior command refers to the result obtained by the downstream controller converting the trajectory point sequence into control quantities recognizable by the vehicle's execution layer. The downstream controller is the control module located between the planning module and the actuator, typically implemented using a proportional-integral-derivative (PID) controller or a model predictive controller. A PID controller is a feedback controller that calculates the control quantity based on a linear combination of the current error, the error integral, and the error derivative. A model predictive controller is an optimization controller that calculates control commands by solving a finite-time optimal control problem online.

[0034] A simulated vehicle is a three-dimensional virtual entity created in a simulation platform that represents the vehicle controlled by the autonomous driving model under test. This virtual entity has the same dynamic parameters as a real vehicle, including vehicle mass, wheelbase, track width, tire model parameters, and suspension characteristic parameters. Configuring an autonomous driving model into a simulated vehicle means binding the control outputs of the model under test to the throttle, brake, and steering execution interfaces of the simulated vehicle, so that the driving behavior commands output by the model can directly drive the movement of the simulated vehicle.

[0035] In step S300 of some embodiments, based on a preset prior model of real traffic participant behavior, behavioral reasoning processing is performed on the driving behavior currently performed by the simulated vehicle and the real-time traffic status fed back by the simulation platform to obtain motion response commands for multiple traffic participants, and the motion status of the multiple traffic participants is updated according to the motion response commands.

[0036] Understandably, a priori model of real traffic participant behavior refers to a machine learning model trained using interactive trajectory data collected from real traffic scenarios. The core function of this model is to take into account the driving behavior of the simulated vehicle at the current moment, the motion state of surrounding traffic participants, and their relative positional relationships with the simulated vehicle, and output the acceleration and steering angle commands for each traffic participant at the next moment.

[0037] It should be noted that behavioral priors refer to the intrinsic knowledge representations learned by the model from a large amount of real-world interaction data, reflecting the typical behavioral patterns and movement laws of human drivers or other traffic participants in specific traffic situations. Machine learning models, on the other hand, are mathematical models that automatically learn the input-output mapping relationship from sample data through a data-driven approach, distinguishing them from methods based on manual rules or physical models.

[0038] The real-time traffic status reported by the simulation platform refers to the snapshot data of the current traffic environment summarized and output by the simulation engine within each simulation step. This includes information such as the position coordinates, velocity vectors, acceleration vectors, orientation angles, lane indices, and relationships with traffic light status for all traffic participants. The simulation step refers to the time interval between two state updates by the simulation engine, typically set to 0.05 seconds, corresponding to a simulation update frequency of 20Hz.

[0039] Furthermore, behavioral reasoning processing refers to the process of using the current driving behavior commands executed by the simulated vehicle and the real-time traffic status fed back by the simulation platform as model inputs, and then performing forward computation to obtain the motion response commands of each traffic participant within one or more future simulation steps. Forward computation refers to the process of propagating the input data layer by layer along the positive direction of each layer of the neural network, sequentially performing mathematical operations such as weighted summation and activation function mapping, and finally generating the output results. Motion response commands include acceleration commands and steering angle commands. Acceleration commands refer to the expected acceleration value of the traffic participant along its current direction of travel; positive values ​​indicate acceleration, and negative values ​​indicate deceleration. Steering angle commands refer to the expected deflection angle of the front wheels of the traffic participant, used to control changes in its driving direction.

[0040] Updating the motion state of multiple traffic participants based on motion response commands involves inputting acceleration and steering angle commands obtained from model inference into the simulation vehicle's kinematics or dynamics module. Updated motion state parameters, such as position, velocity, and orientation, are then calculated through numerical integration, and the simulation engine refreshes the state. Numerical integration refers to a computational method that approximates the solution of continuous motion differential equations using discrete-time stepping. Commonly used methods include the Euler method and the Runge-Kutta method. The Euler method is a first-order numerical integration method that directly calculates the state value at the next moment based on the current state value and rate of change. The Runge-Kutta method is a higher-order numerical integration method that improves integration accuracy by sampling the rate of change multiple times within a single step.

[0041] This embodiment implements a closed-loop interaction mechanism in a simulation environment. Closed-loop interaction means that the driving behavior of the simulated vehicle will affect the motion decisions of surrounding traffic participants, and the changes in the motion of surrounding traffic participants will be fed back to the real-time traffic status at the next moment through the simulation platform, thereby affecting the subsequent perception input and driving decisions of the simulated vehicle, forming a continuously feedback interactive closed loop.

[0042] In step S400 of some embodiments, when the simulated vehicle interacts with the multiple traffic participants for a preset duration, the driving performance of the autonomous driving model is evaluated in a multi-dimensional manner based on multiple evaluation dimensions to obtain the evaluation result of the autonomous driving model.

[0043] It is understandable that the preset duration refers to the pre-defined evaluation period or evaluation mileage. The evaluation duration can be set from 30 minutes to several hours, depending on the complexity of the test scenario and the evaluation objectives. The evaluation mileage can be set from 50 kilometers to several hundred kilometers.

[0044] Continuous interaction refers to the continuous looping execution of steps S200 and S300 within a preset time period. The simulated vehicle generates new driving behavior commands and updates its own motion state based on continuously injected vehicle-road cooperative perception information. Simultaneously, other traffic participants continuously update their own motion states based on prior behavior models. The entire process runs uninterrupted until the preset time period is reached or an early termination condition is triggered. Early termination conditions include a collision, the simulated vehicle crossing the road boundary, or reaching a preset task failure criterion.

[0045] Furthermore, multiple evaluation dimensions refer to an indicator system that comprehensively evaluates autonomous driving models from multiple perspectives, such as safety, compliance, collaboration, comfort, and robustness. Each evaluation dimension corresponds to one or more specific quantitative indicators. Multidimensional evaluation processing involves continuously recording driving events and statistical data related to each evaluation dimension throughout the interaction process, calculating the indicator values ​​for each dimension after the evaluation, and then forming a comprehensive evaluation result through weighted summation or hierarchical evaluation. Weighted summation involves assigning weight coefficients to different evaluation dimensions, multiplying the indicator values ​​of each dimension by their corresponding weights, and then summing them. The weight coefficients reflect the importance of different dimensions in the evaluation objective. Hierarchical evaluation involves setting pre-defined thresholds for each dimension's indicators and providing a hierarchical evaluation conclusion based on whether each dimension meets the thresholds.

[0046] The evaluation results can be output as a structured evaluation report, which includes individual scores for each evaluation dimension, overall scores, timestamps and location information of key events, and diagnostic analysis text for the evaluation results.

[0047] The method provided in this embodiment drives the configuration of the simulation platform through real traffic scenario data, ensuring that the road scenes and traffic flow in the simulation environment are highly consistent with the real world. By injecting vehicle-road cooperative perception information that includes both vehicle-side and roadside perspectives into the autonomous driving model, the simulation platform can support the testing requirements of autonomous driving models with vehicle-road cooperative capabilities. By performing behavioral reasoning processing based on the prior model of real traffic participants' behavior and updating the motion state of traffic participants, other traffic participants can dynamically interact and respond to the driving behavior of the tested model during the simulation. Through multi-dimensional evaluation processing, the autonomous driving model is comprehensively evaluated from multiple perspectives such as safety, compliance, cooperation, comfort, and robustness. Combining the above technical means, this method significantly improves the consistency between the simulation evaluation results of the autonomous driving model and its real-world driving performance.

[0048] In some embodiments of the present invention, firstly, the road structure data, traffic participant trajectory data, traffic signal status data, vehicle sensor data, and roadside perception data collected synchronously at real traffic intersections based on the same time reference are subjected to spatiotemporal alignment processing to obtain spatiotemporal synchronized data.

[0049] Understandably, a common time reference means that all participating sensor and positioning devices use the standard time provided by a Global Navigation Satellite System (GNSS) as a unified time source for data tagging. The GNSS can be any one of GPS, BeiDou, GLONASS, or Galileo systems, or a combination thereof. Tagging refers to appending the current system time as a timestamp to the metadata field of each sensor data frame as it is generated. The timestamp typically has millisecond precision to ensure that the alignment accuracy of the sensor data in the time dimension meets the requirements of simulation reconstruction.

[0050] The methods for collecting road structure data include, but are not limited to: using a surveying vehicle equipped with a high-precision global positioning system and an inertial measurement unit to drive in the target intersection area and collect three-dimensional point cloud data and panoramic image data of the road surface; or directly obtaining vectorized high-precision map data of the target intersection from a high-precision map service provider.

[0051] The methods for collecting trajectory data of traffic participants include: performing target detection and multi-target tracking processing on point cloud data collected by roadside LiDAR to obtain a continuous trajectory sequence for each traffic participant; or using video data collected by intersection surveillance cameras to perform deep learning-based visual target detection and tracking, obtaining the trajectory in the pixel coordinate system, and then converting it into a physical trajectory in the world coordinate system through camera calibration parameters. The methods for collecting traffic signal status data include: directly reading the phase status of traffic lights through the signal output interface of the intersection traffic signal controller, or detecting the traffic light status from roadside camera images using image recognition methods. The methods for collecting vehicle-mounted sensor data include: installing a forward-looking camera, a surround-view camera group, and a multi-beam LiDAR on the vehicle to collect raw images and point cloud data at a fixed frequency. The methods for collecting roadside perception data include: installing a fixed-angle wide-angle camera and LiDAR on intersection poles or buildings to collect raw perception data of the entire intersection view at a fixed frequency.

[0052] Furthermore, spatiotemporal alignment refers to the operation of unifying the original data from different data sources, sampling frequencies, and coordinate systems to the same time axis and spatial coordinate reference system. The specific implementation of time alignment involves using the Global Navigation Satellite System (GNSS) time as the reference time axis and interpolating or resampling the data frames from each data source according to their timestamps, so that all data are aligned to the same sampling time sequence in the time dimension. Interpolation refers to calculating an approximate data value for a time point where there is no corresponding data between two sampling times through linear interpolation or spline interpolation of adjacent data. Resampling refers to converting the original non-uniformly sampled time series data into a uniform time series with a fixed sampling rate through interpolation and extrapolation methods.

[0053] The specific implementation of spatial alignment involves uniformly transforming the different local coordinate systems containing data from different sensors to a global world coordinate system with the intersection center as the origin, using calibration parameters and a coordinate transformation matrix. Calibration parameters include the installation position, installation angle, and intrinsic parameter matrix of each sensor. The intrinsic parameter matrix refers to a parameter matrix describing the internal optical or measurement characteristics of the sensor, such as camera focal length, principal point coordinates, and distortion coefficients, or the number of lines and vertical angular resolution of a lidar sensor.

[0054] Then, the sensor data of the same traffic participant generated by different sensors in the spatiotemporal synchronization data are correlated and labeled to obtain multi-source data packets.

[0055] It is understandable that sensor data generated by the same traffic participant under different sensors refers to multiple sets of sensor data records generated when the same vehicle, non-motorized vehicle, or pedestrian entity is observed simultaneously by vehicle-mounted sensors and roadside sensors deployed at different locations and from different perspectives at the same or near moment. For example, a white sedan turning left at an intersection is simultaneously detected by four different sensors: the vehicle's forward-facing camera, the vehicle's lidar, the roadside east-facing camera, and the roadside north-facing lidar, generating four sets of sensor data from different perspectives.

[0056] Furthermore, association labeling refers to identifying and labeling all sensor data fragments belonging to the same traffic participant entity using data association algorithms, assigning each fragment a unique entity identifier, enabling the association retrieval of observation data from different sensors for that entity through the entity identifier. Data association algorithms can employ multi-target association methods based on spatial location. This involves constructing an association cost matrix by calculating the Euclidean or Mahalanobis distance between targets based on the spatial coordinates of targets detected by each sensor at the same time, and then using the Hungarian algorithm or the global nearest neighbor algorithm to solve for the optimal association matching result. Euclidean distance refers to the straight-line distance between two spatial points in a Cartesian coordinate system. Mahalanobis distance is a normalized distance metric considering the covariance of the data distribution. The association cost matrix is ​​a matrix with the targets detected by each sensor as rows and columns, and the association cost as elements. The Hungarian algorithm is a combinatorial optimization algorithm for solving the optimal matching problem in a bipartite graph.

[0057] After the association annotation process is completed, a multi-source data packet is obtained. This multi-source data packet has a spatiotemporal label containing a unified timestamp and spatial coordinate reference system, which enables the subsequent simulation platform configuration process to directly use various types of data in the data packet for scene reconstruction, traffic flow generation and sensor configuration.

[0058] This embodiment establishes multi-dimensional spatiotemporal data associations for real traffic intersections through spatiotemporal alignment and association annotation processing, ensuring temporal and spatial consistency among data in multi-source data packets, and laying a data foundation for the high-fidelity configuration of the subsequent simulation platform.

[0059] In some embodiments of the present invention, firstly, the road structure data in the multi-source data packets is processed by automated road network generation to obtain an initial digital twin static scene.

[0060] Automated road network generation refers to the process of automatically constructing a three-dimensional road network model that can be loaded onto a simulation platform using computer programs based on road structure data. A specific implementation method includes the following sub-steps: The first step is to extract coarse-grained road topology and building outline data for the target intersection from open street map data sources. Open street map data sources refer to platforms like OpenStreetMap that provide open-source geographic information data. This data is organized using three basic elements: nodes, paths, and relationships. Nodes represent geographic locations, paths represent ordered sequences of nodes along road or area boundaries, and relationships represent the logical connections between multiple elements. Coarse-grained road topology describes which roads are included at the intersection, the connectivity between these roads, and the type of intersection. Building outline data refers to the polygonal outlines of the base surfaces of buildings surrounding the intersection and their floor heights.

[0061] The second step involves using high-precision map editing tools to generate detailed road structure elements from the high-precision map. These tools, such as RoadRunner, are software specifically designed for editing high-precision maps and constructing road networks for autonomous driving scenarios. A high-precision map contains lane-level precision in its geometric and topological information, typically down to the centimeter level. It includes precise lane line locations, stop line locations, connections between lanes at intersections, and the positions of traffic signs and lights. Detailed road structure elements are extracted from or edited from the high-precision map, including 3D geometric models of lane lines, stop line markings, pedestrian crossing markings, and detailed models of connecting arcs within intersections.

[0062] The third step involves aligning the coarse-grained road topology and building outline data with the refined road structure elements using satellite images from the road structure data as a spatial reference. This process is then performed in a 3D scene fusion editing tool, which refers to open-source 3D content creation software such as Blender that has functions for importing and exporting 3D models, performing spatial transformation operations, and editing materials.

[0063] Spatial coordinate alignment refers to unifying 3D models from three different sources—open street map data sources, high-precision map editing tools, and satellite imagery—under the same geographic coordinate system. The alignment operation uses visible ground features in the satellite imagery as control points, calculating coordinate transformation parameters through affine or similarity transformations to transform each model to a unified coordinate system. Affine transformation refers to linear spatial transformations that include four basic operations: translation, rotation, scaling, and shearing. Model fusion processing involves geometrically overlaying and seam-processing the spatially aligned coarse-grained building models, refined road models, and other layers of models. This eliminates geometric discontinuities and texture mismatches between models, creating a visually and geometrically unified initial 3D scene.

[0064] Then, the initial digital twin static scene is manually modified to obtain the modified digital twin static scene, and the modified digital twin static scene is loaded into the simulation platform.

[0065] Understandably, manual model correction refers to the manual inspection and detail adjustment of the automatically generated initial scene by an operator within the graphical user interface of a 3D scene fusion and editing tool. This operation includes two aspects: The first step is adding traffic-related facility models. These models include at least streetlight models and roadside sensing device pole models. Streetlight models are 3D models with dimensions identical to actual streetlights at intersections, while roadside sensing device pole models are 3D models with dimensions identical to the actual poles used to install roadside cameras and LiDAR at intersections. Adding a model involves selecting the corresponding facility model from the model library and dragging and dropping it to the appropriate location in the scene.

[0066] The second step involves manually adjusting the position and orientation of the added traffic ancillary facility models. Position adjustment refers to fine-tuning the model's spatial coordinates to match its actual location, while orientation adjustment involves fine-tuning the model's pitch, yaw, and roll angles to align its orientation with the actual installation direction. The accuracy of these manual adjustments is based on high-precision maps or on-site measurement photographs.

[0067] Finally, the corrected digital twin static scene is exported as a scene description file in a simulation platform-compatible format and imported into the simulation platform. The export process includes exporting the 3D model geometry data as FBX or OBJ format mesh files and exporting the road network topology information as an OpenDRIVE format road network description file. OpenDRIVE format is an open standard data format for describing road networks, supporting the expression of lane-level road geometry and topology. FBX format is a universal format for exchanging 3D model data, supporting mesh geometry, materials, textures, and animation information. After importing the scene description file into the simulation platform, the platform parses the road geometry data and scene model data in the file, reconstructing a virtual simulation scene in the simulation engine that is highly consistent with the real intersection.

[0068] This embodiment combines automated road network generation with manual model correction, which can efficiently build the scene skeleton using automated tools, while ensuring that the scene details are highly consistent with the real intersections through manual correction, thus achieving a balance between scene construction efficiency and fidelity.

[0069] In some embodiments of the present invention, the trajectory data of traffic participants in the multi-source data packet are processed by real trajectory playback to obtain a first type of traffic flow segment.

[0070] Understandably, real-world trajectory playback processing refers to directly extracting complete or partial trajectory segments from real-world traffic participant trajectory data, preserving the position coordinates and velocity sequences of each trajectory point in real time and space, without any modification or generation of new trajectories. During extraction, filtering can be performed according to a preset set of driving behavior types.

[0071] Furthermore, the preset driving behavior type set refers to a predefined list of driving behavior categories with typical testing value, including left turns at intersections, right turns at intersections, U-turns, following other vehicles, lane changes, yielding to pedestrians, and waiting at traffic lights. For example, a left turn refers to the complete trajectory of a traffic participant entering an intersection from the approach lane, traveling along the left turn path, and exiting from the exit lane of the intersecting road; following other vehicles refers to a continuous trajectory segment in which a traffic participant follows the vehicle in front within the same lane while maintaining a safe distance; and changing lanes refers to the process trajectory of a traffic participant crossing lane lines from the current lane to enter an adjacent lane.

[0072] The filtering operation refers to traversing all collected trajectory data, automatically identifying and marking the driving behavior type of the trajectory segments based on the geometric features of the trajectory and its relative position to the intersection area, and extracting the trajectory segments that meet the requirements.

[0073] Simultaneously, data-driven trajectory generation processing is performed on the trajectory data of traffic participants to obtain the second type of traffic flow segments.

[0074] It is understandable that data-driven trajectory generation processing refers to training a trajectory generation model using real-collected trajectory data, and then calling the model to generate a new driving trajectory that has the same statistical distribution as the real data under given initial conditions and environmental constraints.

[0075] Trajectory generation models can be implemented using trajectory generation models based on generative adversarial networks or trajectory generation models based on variational autoencoders. A generative adversarial network is a generative model framework consisting of two neural networks: a generator and a discriminator. The generator is responsible for generating realistic data samples from random noise, while the discriminator is responsible for distinguishing between real samples and generated samples. The two improve each other through adversarial training, ultimately enabling the generator to generate data that is indistinguishable from real data.

[0076] It should be noted that a variational autoencoder is a generative model based on variational inference, consisting of an encoder and a decoder. The encoder maps the input data to probability distribution parameters in the latent space, and the decoder reconstructs new data consistent with the distribution of the input data from the sampling points in the latent space. In trajectory generation tasks, the input of this model is the starting position, the target position, and the environmental traffic state encoding, and the output is a complete sequence of motion trajectories.

[0077] Then, the first type of traffic flow segment and the second type of traffic flow segment are subjected to spatiotemporal fusion processing to obtain a dynamic traffic flow composed of multiple traffic participants.

[0078] As can be understood, spatiotemporal fusion processing refers to combining and arranging multiple trajectory segments from real-world playback and data-driven generation according to their respective timestamp and spatial location information within a unified time axis and spatial coordinate system. This forms a complete traffic flow scenario data containing the movement trajectories of multiple traffic participants at different times and locations. The fusion processing process requires handling the temporal sequence and spatial start-end connections of different trajectory segments to ensure that all trajectories cover the target simulation period in time and the target simulation area in space. The target simulation period refers to the planned duration of a simulation evaluation task, for example, 900 seconds from the start to the end of the simulation. The target simulation area refers to the spatial range of the simulation scenario of interest in this evaluation, typically a radius of several hundred meters centered on the target intersection.

[0079] Finally, based on dynamic traffic flow, the simulation platform is configured with traffic participant instances and environmental parameters.

[0080] Understandably, traffic participant instance configuration refers to creating a corresponding 3D solid model and setting its initial state in the simulation platform based on the trajectory data and attribute information of each traffic participant. Specifically, this involves parsing the type identifier, geometric dimensions, appearance model identifier, and initial pose contained in the traffic participant trajectory data to obtain instance attribute configuration parameters. The type identifier refers to an enumerated value or label that distinguishes different types of traffic participants, such as motor vehicles, non-motor vehicles, and pedestrians. Geometric dimensions refer to the length, width, and height of the traffic participant's 3D bounding box.

[0081] The appearance model identifier refers to the index number or resource path of the corresponding 3D appearance model of the traffic participant in the simulation platform's model library. For example, the blue sedan model corresponds to the resource number car_sedan_blue_01. The initial pose refers to the 3D position coordinates and orientation angle of the traffic participant in the scene at the start of the simulation. Based on the instance attribute configuration parameters, the simulation platform calls the corresponding interface to generate multiple 3D solid model instances of the traffic participants and places them at the specified initial poses.

[0082] Environmental parameter configuration refers to setting the environmental rendering conditions such as weather and lighting in the simulation scene. Specifically, this involves parsing and processing the weather type, light intensity value, and time period information recorded in the trajectory data of traffic participants to obtain environmental simulation parameters. Weather types include categories such as sunny, cloudy, overcast, light rain, moderate rain, heavy rain, snow, and fog. Light intensity value refers to the overall illuminance level of the scene. Time period information refers to the time period in which the trajectory data was collected, such as early morning, morning, noon, afternoon, evening, and night. Based on the environmental simulation parameters, the weather particle system and light source parameters of the simulation platform are configured. The weather particle system refers to the particle system in the simulation engine used to generate and render atmospheric particle effects such as raindrops, snowflakes, and fog particles. Configuration includes parameters such as particle emission rate, particle size, particle falling speed, and particle density. Light source parameters refer to the set of parameters controlling the main directional light source in the scene, including the solar azimuth angle, solar elevation angle, light source color temperature, and light source intensity.

[0083] This embodiment constructs dynamic traffic flow by combining real trajectory playback with data-driven trajectory generation. It not only retains typical driving behaviors in real scenarios but also generates diverse new driving scenarios. Through fine configuration of traffic participant instances and environmental parameters, the simulation environment is highly consistent with the real scenario at both the visual and physical levels.

[0084] In some embodiments of the present invention, spatiotemporal overlap detection processing is performed on the first type of traffic flow segment and the second type of traffic flow segment to determine the conflict trajectory segment with spatiotemporal conflict.

[0085] Understandably, spatiotemporal overlap detection processing refers to detecting whether there is spatiotemporal overlap or excessive proximity between the trajectories of different traffic participants under a unified time axis and spatial coordinate system.

[0086] In practice, the trajectory of each traffic participant is represented as a sequence of trajectory points with timestamps and spatial coordinates. The trajectory points of all traffic participants are traversed point by point in time, and the spatial distance between any two different traffic participants is calculated at the same timestamp. When this spatial distance is less than a preset safe distance threshold, a spatiotemporal conflict is determined between the two traffic participants at that moment. The safe distance threshold is dynamically determined based on the type and speed of the traffic participants; for example, 3 meters can be used between motor vehicles, and 2 meters between a motor vehicle and a pedestrian. A spatiotemporal conflict refers to two or more traffic participants occupying excessively close spatial positions at the same time. In the real world, this situation usually leads to collisions or dangerous proximity. Conflict trajectory segments refer to those local trajectory segments where spatiotemporal conflicts are detected, including the start and end time range of the conflict and the range of trajectory point indices involved.

[0087] Then, conflict resolution processing is performed on the conflict trajectory segments to obtain dynamic traffic flow without spatiotemporal conflicts.

[0088] Understandably, conflict resolution refers to applying adjustment operations to detected conflict trajectory segments to eliminate spatiotemporal conflicts between trajectories, making the entire traffic flow a physically plausible motion sequence. The goal of the adjustment operation is to ensure that all traffic participants maintain a safe distance at any given time, while preserving the original trajectory behavior characteristics as much as possible. After conflict resolution, a dynamic traffic flow without spatiotemporal conflicts is obtained, which can be directly used for subsequent simulation platform configuration and processing.

[0089] This embodiment, by adding spatiotemporal conflict detection and resolution processing before configuration processing, can ensure that there are no unreasonable physical overlaps or penetrations between traffic participants in the fused dynamic traffic flow, thereby improving the realism and continuity of the simulation process.

[0090] In some embodiments of the present invention, the step of performing conflict resolution processing on the conflict trajectory segments to obtain the dynamic traffic flow without spatiotemporal conflicts includes: The velocity difference sequence is obtained by performing difference calculation on the velocity sequences of adjacent trajectories in the conflict trajectory segment; The velocity difference sequence is smoothed by filtering to obtain a smoothed velocity sequence. Based on the smoothed velocity sequence, the conflict trajectory segment is resampled to obtain a smoothed trajectory segment. For the trajectories that still have spatiotemporal conflicts after trajectory resampling in the conflict trajectory segments, local path replanning is performed to obtain the replanned trajectory segments. The trajectory segments after smooth transition and the trajectory segments after replanning are spliced ​​together to obtain the dynamic traffic flow without spatiotemporal conflicts.

[0091] Understandably, adjacent trajectories refer to the trajectories corresponding to two traffic participants that conflict in space and time. A velocity sequence is a one-dimensional array consisting of the instantaneous velocity values ​​of a trajectory at each sampling point, arranged chronologically. Difference calculation involves taking the velocity values ​​of two adjacent trajectories at their corresponding time points and calculating the difference, resulting in a velocity difference sequence. The velocity difference sequence reflects the degree of mismatch in the motion states between the two conflicting traffic participants; the sign and absolute value of the difference indicate the severity and direction of the conflict.

[0092] Furthermore, smoothing filtering refers to using filtering algorithms in digital signal processing to eliminate high-frequency abrupt changes and spikes in the velocity difference sequence, making the velocity change trend smoother and more continuous.

[0093] One specific implementation method is to use a one-dimensional Gaussian filter for smoothing. A Gaussian filter is a linear smoothing filter based on a Gaussian function as the convolution kernel. The kernel weights are determined by the Gaussian function, with the largest weight at the center and gradually decreasing towards the sides. The standard deviation parameter of the Gaussian function controls the degree of smoothing; the larger the standard deviation, the stronger the smoothing effect, but the less detail is preserved.

[0094] Another implementation method is to use a moving average filter, which involves sliding a fixed-width window across the speed difference sequence and taking the arithmetic mean of the data within the window as the smoothed value at the window's center point. The window width of the moving average filter is an adjustable parameter, typically ranging from 5 to 15 sampling points. After smoothing filtering, a smoothed speed difference sequence is obtained. Then, based on the original speed sequence and the speed difference sequence, the adjusted speed sequence of each traffic participant—that is, the smoothed speed sequence—can be calculated.

[0095] Furthermore, trajectory resampling refers to recalculating the position coordinates of traffic participants at each trajectory sampling point using the smoothed velocity sequence as motion constraints. Specifically, starting from the beginning of the conflict segment, the velocity value at each moment in the smoothed velocity sequence is used as the average velocity between that moment and the next moment. This average velocity is multiplied by the sampling time interval to obtain the displacement increment. This displacement increment is then accumulated along the travel direction of the previous moment to the position coordinates of the previous moment, and this process is repeated point by point to obtain the entire resampled trajectory. After trajectory resampling, a smoothed trajectory segment is obtained, achieving a smooth transition in velocity and eliminating the abrupt velocity changes in the original conflict.

[0096] Furthermore, for trajectories that still have spatiotemporal conflicts after trajectory resampling in conflict trajectory segments, local path replanning is performed to obtain replanned trajectory segments. Some severe conflicts are difficult to completely eliminate by speed adjustment alone, such as when two trajectories intersect in space and the speed adjustment margin is insufficient. In this case, local replanning of the spatial path of the trajectory is required.

[0097] Local path replanning refers to regenerating a new path that avoids the conflict zone within the feasible space using a path search algorithm, without changing the constraints of the traffic participant's origin and destination in that segment. A typical implementation uses the A* algorithm for path search. The A* algorithm is a heuristic graph search algorithm whose core is to maintain a set of open nodes and a set of closed nodes in the state space graph. Each time, the node with the smallest cost function value is selected from the open set for expansion. The cost function is defined as the sum of the actual cost from the origin to the current node and the heuristically estimated cost from the current node to the target node. The heuristically estimated cost is usually the Euclidean distance from the current node to the target node. The feasible space for path search is the safe zone surrounding the conflict zone.

[0098] Another implementation method employs a spline curve-based path smoothing approach. Control points are selected on both sides of the conflict area, and a new, smooth path is generated around the conflict area using either a Bézier curve or a B-spline curve. A Bézier curve is a smooth parametric curve defined by control polygons. A B-spline curve is a piecewise polynomial curve using B-spline basis functions, exhibiting local support; that is, modifying a control point only affects the local shape of the curve. After local path replanning, the replanned trajectory segment is obtained.

[0099] Furthermore, trajectory splicing refers to connecting the various trajectory segments that have undergone conflict resolution with normal trajectory segments that have not detected conflicts, according to their original chronological order, to form a complete and continuous time-series trajectory. The splicing operation includes interpolating and completing the trajectory points at the splicing points to ensure that the trajectory at the splicing point is continuous and without abrupt changes in spatial position and velocity. After trajectory splicing is completed, a dynamic traffic flow without spatiotemporal conflicts is obtained.

[0100] This embodiment adopts a hierarchical processing approach. It first attempts to resolve minor conflicts by adjusting speed smoothly, and then performs path replanning for serious conflicts that cannot be resolved by speed adjustment. While preserving the original trajectory behavior characteristics as much as possible, it ensures that the entire traffic flow is physically feasible and safe.

[0101] In some embodiments of the present invention, the vehicle-road cooperative perception information includes at least a vehicle-side view data stream and a roadside view data stream.

[0102] Understandably, the vehicle-side perspective data stream refers to the sequence of perception data generated and output by virtual sensors simulated and installed on a simulated vehicle. This data stream takes the simulated vehicle as the center of the first-person perspective and provides perception information about the local environment around the vehicle.

[0103] The vehicle-side view data stream may include at least one of image data stream and point cloud data stream. The image data stream refers to a two-dimensional image sequence generated and output frame by frame by a virtual camera. Each frame is a two-dimensional matrix consisting of pixel rows and pixel columns. Each pixel contains the brightness value of three color channels: red, green, and blue. The brightness value of each channel is usually represented by an integer range from 0 to 255.

[0104] A virtual camera refers to the computational module in the simulation engine that simulates the optical imaging process of a real camera. Its core is the perspective projection model. The perspective projection model maps spatial points in the three-dimensional world coordinate system to pixel coordinates on a two-dimensional image plane through camera intrinsic and extrinsic parameter matrices. The camera intrinsic parameter matrix includes parameters such as focal length, principal point coordinates, and distortion coefficients. The camera extrinsic parameter matrix includes the camera's rotation matrix and translation vector in the world coordinate system. A point cloud data stream refers to a sequence of three-dimensional point clouds generated and output frame by frame by the virtual LiDAR. Each frame of the point cloud is a collection of unordered three-dimensional spatial points, and each point contains three-dimensional coordinates and reflection intensity information. A virtual LiDAR refers to the computational module in the simulation engine that simulates the ranging principle of a real LiDAR. Its core is the ray casting algorithm. The ray casting algorithm starts from the virtual emission point of the LiDAR, emits rays along the directions of each laser channel, calculates the position of the first intersection point between the ray and the scene's geometric model, uses the intersection point coordinates as the point cloud data point for that channel, and calculates the reflection intensity value based on the material properties of the model at the intersection point.

[0105] Furthermore, the roadside view data stream refers to the sequence of perception data generated by virtual sensors simulated and installed on roadside poles or buildings. This data stream uses the roadside deployment location as the observation point and provides perception information of the global field of view at the intersection, covering blind spots that vehicle-mounted sensors cannot observe due to obstruction or field-of-view limitations. The roadside view data stream can also include at least one of image data streams and point cloud data streams.

[0106] Furthermore, compared to vehicle-side view data streams, roadside view data streams have fixed observation positions and a wider field of view, typically employing wide-angle lenses or top-down deployment methods. A bird's-eye view refers to a deployment method where the virtual sensor's optical axis is vertically or nearly vertically downward, looking down at the entire intersection area from above. A global intersection view refers to a deployment method where virtual sensors are deployed on high poles at the corners of the intersection, covering the entire interior area of ​​the intersection at an angle.

[0107] Vehicle-road cooperative perception information is obtained based on multi-source data packets, and its specific generation method includes the following processing steps: First, based on the vehicle sensor data in the multi-source data packet, sensor configuration processing is performed on the virtual vehicle sensor to obtain the configured virtual vehicle sensor. Sensor configuration processing refers to setting the corresponding parameters of the virtual sensor bound to the simulated vehicle model in the simulation platform according to the installation position, installation orientation, field of view, image resolution, and frame rate of the real vehicle sensor recorded in the multi-source data packet. The installation position is described in three-dimensional coordinates within the vehicle's local coordinate system. This local coordinate system typically uses the projection point of the vehicle's rear axle center onto the ground as the origin, the vehicle's forward direction as the positive X-axis, the left side of the vehicle as the positive Y-axis, and the vertical upward direction as the positive Z-axis. The installation orientation is described by three Euler angles: pitch, yaw, and roll. The field of view is described by both horizontal and vertical angles, representing the sensor's observable range. The image resolution is described by the number of horizontal and vertical pixels, representing the output image size of the virtual camera. The frame rate is described in Hertz (Hz), representing the number of frames the sensor acquires and outputs per second. After configuration, you will get the configured virtual vehicle sensor, whose parameter settings are consistent with those of the actual vehicle sensor used for data collection.

[0108] Then, the configured virtual vehicle sensors undergo data generation processing to obtain the vehicle-side view data stream. Data generation processing refers to the process by which the simulation engine calculates the virtual sensor output data in real time each frame based on the position and material of the 3D model in the current scene, through the rendering pipeline, during simulation operation. For virtual cameras, the rendering pipeline includes stages such as vertex shading, rasterization, and pixel shading, ultimately outputting a 2D image that conforms to the set resolution and frame rate. For virtual LiDAR, the rendering pipeline includes stages such as ray generation, ray intersection with the scene, and intersection point attribute calculation, ultimately outputting a 3D point cloud containing hundreds of thousands to millions of points per frame.

[0109] Simultaneously, based on the roadside perception data in the multi-source data packet, sensor configuration processing is performed on the virtual roadside sensor to obtain the configured virtual roadside sensor. This configuration process is similar to that of virtual vehicle-mounted sensors, the difference being that the configuration parameters are derived from the installation location, orientation, field of view, and resolution parameters of the real roadside sensor recorded in the multi-source data packet. The installation location of the roadside sensor is at the top of the roadside pole or high up on a building, described using three-dimensional coordinates in the global world coordinate system.

[0110] Then, the configured virtual roadside sensors are processed to generate roadside view data streams. This data generation process is consistent with the vehicle-side data generation process in principle, both being generated in real time by the simulation engine. The difference lies in the virtual sensor parameters and viewpoint positions used for rendering, which result in different observation angles and coverage areas of the generated images or point cloud data compared to the vehicle-side data streams.

[0111] This embodiment, by configuring and generating vehicle-side and roadside perspective data streams respectively, can realistically simulate the generation process of multi-view perception information in a vehicle-road cooperative system in a simulation environment, providing a data foundation for subsequent information injection.

[0112] In some embodiments of the present invention, injecting vehicle-road cooperative perception information into the autonomous driving model to obtain driving behavior instructions includes the following processing steps: First, the vehicle-side view data stream and the roadside view data stream are time-stamped and synchronized to obtain a synchronized vehicle-road cooperative data stream. Time-stamp synchronization refers to pairing or aligning data frames from two independently generated data streams, each carrying its own timestamp, according to the same or closest timestamps.

[0113] In practice, using the timeline of the vehicle-side data stream as the reference timeline, for each frame in the vehicle-side data stream, the frame with the closest timestamp in the roadside data stream is selected as its synchronization frame. "Closest" is defined as having the smallest absolute value of the timestamp difference, and the difference does not exceed a preset synchronization tolerance threshold, which is typically set to half a simulation step. The successfully paired vehicle-side and roadside data frames are combined into a single synchronization frame data unit. Multiple synchronization frame data units are arranged in chronological order to form the synchronized vehicle-road cooperative data stream.

[0114] Then, the synchronized vehicle-to-infrastructure (V2I) data stream undergoes protocol encapsulation processing to obtain a message queue conforming to the preset V2I communication protocol. Protocol encapsulation processing refers to packaging and encapsulating the synchronized data according to a predefined communication protocol format to generate structured messages that are easy to transmit and parse. The preset V2I communication protocol refers to a predefined data exchange format specification used to define the data organization method of vehicle-side sensing data and roadside sensing data during cooperative transmission. A typical protocol encapsulation format includes a frame header field, a timestamp field, a data source identifier field, a data length field, a payload data field, and a checksum field. The frame header field identifies the start of the message and the protocol type; the timestamp field records the acquisition time of the frame data; the data source identifier field distinguishes whether the frame data comes from the vehicle or the roadside; the data length field indicates the byte length of the payload data; the payload data field carries the actual sensing data content; and the checksum field is used by the receiving end to verify data integrity. A message queue is a first-in, first-out (FIFO) linear data structure. Newly generated messages are added from the tail of the queue, and the receiver retrieves messages from the head of the queue for processing. The message queue mechanism can realize asynchronous and decoupled communication between data producers and data consumers.

[0115] Then, the message queue is sent to the autonomous driving model in real time to inject vehicle-road cooperative perception information. Real-time sending means that each message frame in the message queue is sent to the autonomous driving model immediately after its generation via network or inter-process communication, without caching delay. The communication method depends on the access form of the autonomous driving model. If the autonomous driving model runs in the same process as a plugin or module within the simulation platform, data can be transferred through memory sharing or function callbacks. If the autonomous driving model runs as an independent process or independent container, data can be transferred through socket communication or remote procedure call middleware. Socket communication refers to a network programming interface based on Transmission Control Protocol (TCP) or User Datagram Protocol (UDP). TCP provides connection-oriented reliable data transmission, while UDP provides connectionless best-effort data transmission. Remote procedure call middleware refers to a robot operating system or other distributed communication framework that provides a message passing mechanism based on a topic publish-subscribe pattern.

[0116] Finally, the control commands output by the autonomous driving model based on the message queue are processed to obtain driving behavior commands for driving the simulated vehicle. After receiving the perception data in the message queue, the autonomous driving model calculates and outputs control commands through its internal algorithm. The form of the control commands depends on the output layer design of the autonomous driving model. If the model output directly outputs vehicle control quantities, such as three consecutive values ​​representing normalized steering wheel angle, throttle opening, and braking pressure, then the command processing refers to mapping the normalized values ​​to the range of actual physical quantities acceptable to the vehicle model on the simulation platform.

[0117] For example, normalized steering wheel angle is mapped to a steering wheel angle range, normalized throttle opening is mapped to a throttle opening range, and normalized brake pressure is mapped to a master cylinder pressure range. If the model output is a sequence of desired trajectory points, the instruction processing includes calling the underlying controller to convert the trajectory point sequence into vehicle control quantities. The underlying controller can be a proportional-integral-derivative (PID) controller, which uses the lateral and directional deviations between the current actual vehicle position and the desired trajectory point position as control errors. It calculates the steering wheel angle instruction through a linear combination of proportional, integral, and derivative terms; and calculates the throttle and brake instructions based on the deviation between the current speed and the desired speed.

[0118] The underlying controller can also employ a model predictive controller, which uses the vehicle's kinematics or dynamics model as the predictive model to solve a constrained optimization problem in a finite time domain. The objective function minimizes the tracking error and the change in control input. Constraints include vehicle dynamic limits and actuator saturation limits. The output of the instruction solution processing is a driving behavior command that can be directly executed by the simulated vehicle dynamics model, including steering wheel angle, throttle opening, and braking pressure values.

[0119] This embodiment achieves the standardized injection of vehicle-road cooperative perception information into the autonomous driving model and the accurate generation of driving commands through four processing steps: timestamp synchronization, protocol encapsulation, message queue transmission, and instruction decoding. This ensures the reliability, real-time performance, and scalability of information interaction between the model under test and the simulation platform.

[0120] In some embodiments of the present invention, the training steps of the prior model of real traffic participant behavior include the following processing steps: First, sample construction processing is performed on the interaction trajectory segments between multiple traffic participants extracted from multi-source data packets to obtain a sample dataset.

[0121] Interactive trajectory segments refer to continuous trajectory data segments extracted from multi-source data packets that contain obvious interactive behaviors between two or more traffic participants. Interactive behavior refers to the behavioral changes resulting from the direct influence of one traffic participant's change in motion state on one or more other traffic participants. Typical scenarios include: The following interaction involves a vehicle slowing down in front of you, causing the following vehicle to slow down as well; a lane-changing interaction involves a vehicle from an adjacent lane cutting into your lane, causing your vehicle to slow down and give way; and a conflict interaction involves your vehicle and a vehicle approaching from the opposite direction at an unsignalized intersection taking turns passing according to priority rules.

[0122] The method for extracting interactive trajectory segments involves setting a fixed time window length and sliding the time window across the complete trajectory data with a fixed step size. For the trajectory data within each window, the interaction intensity index between traffic participants is calculated, and windows with interaction intensity exceeding a threshold are selected as interactive trajectory segments. The interaction intensity index can be quantified using metrics such as the reciprocal of the minimum headway, the maximum value of the reciprocal of the collision time, or the peak value of the acceleration cross-correlation function. The minimum headway refers to the time required for the front of the following vehicle to reach the rear of the preceding vehicle, equal to the distance between the two vehicles divided by the speed of the following vehicle. The collision time refers to the time required for both vehicles to continue moving at their current speed and direction until a collision occurs.

[0123] Furthermore, the sample construction process refers to transforming the extracted interaction trajectory fragments into paired data of input features and output labels that can be used for supervised learning training. Each sample consists of two parts: input features and output labels. The input features include the driving behavior of the simulated vehicle at multiple consecutive time points, the motion state of each individual among multiple traffic participants, and the relative positional relationship between each individual among multiple traffic participants and the simulated vehicle.

[0124] The driving behavior of the simulated vehicle refers to its state variables such as speed, acceleration, and heading angle at each sampling time. The motion state of each traffic participant refers to its position coordinates, velocity vector, acceleration vector, and heading angle. The relative positional relationship refers to the longitudinal and lateral distances between the participant's current position and the simulated vehicle's current position. The longitudinal distance is the component along the simulated vehicle's direction of travel, and the lateral distance is the component perpendicular to the simulated vehicle's direction of travel. The output labels are the acceleration and steering angle commands for each participant at the next time step corresponding to the input features. The acceleration command is the expected acceleration value along the participant's current heading direction, and the steering angle command is the expected deflection angle of the participant's front wheels.

[0125] Then, using the sample dataset, the pre-defined graph neural network is trained to obtain the trained model.

[0126] Understandably, graph neural networks are a deep learning model architecture specifically designed for processing graph-structured data. In this application scenario, all traffic participants in the simulation scene are constructed as a spatiotemporal interaction graph. Each node in the graph represents a traffic participant, and the node's feature vector consists of the individual's motion state information at the current moment. Each edge in the graph represents an interaction relationship between two traffic participants, and the edge's feature vector consists of the relative distance and relative speed between the two traffic participants. The interaction relationship can be a fully connected relationship, meaning there is an edge between any two traffic participants; or it can be a locally connected relationship based on a distance threshold, meaning that a connection is established only when the spatial distance between two traffic participants is less than a certain threshold, in order to reduce computational load. The distance threshold is set to 50 meters to 100 meters based on the scene characteristics.

[0127] The core computational mechanism of graph neural networks (GNNs) is message passing. In each layer of the GNN, each node first receives messages passed from all its neighboring nodes via edges. The message content is the result of a linear transformation of the neighboring nodes' feature vectors. Then, all received messages are aggregated, and the aggregation function can be summation, averaging, or taking the maximum value. Finally, the aggregated message is combined with the node's own current feature vector, and the node's own feature vector is updated through a non-linear transformation. After stacking multiple layers of GNNs, each node can gather information from multi-hop neighbors. Multi-hop neighbors refer to neighboring nodes indirectly connected through multiple intermediate nodes. Through multi-layer message passing, the model can capture the complex spatial dependencies and interactive game relationships between multiple traffic participants in a traffic scenario.

[0128] Training involves pairing input features and output labels from the sample dataset and iteratively adjusting the weight parameters of the graph neural network using backpropagation and gradient descent optimization algorithms. This process gradually minimizes the errors between the predicted acceleration and steering angle values ​​of each traffic participant and the actual output labels. The loss function is the mean squared error function, which is the average of the squares of the differences between the predicted and actual values. The optimization algorithm uses the Adam optimizer, an adaptive learning rate stochastic optimization algorithm based on first-order and second-order gradient moment estimation. During training, the sample dataset is divided into three parts: a training set, a validation set, and a test set. The training set is used for updating model parameters, the validation set is used for hyperparameter tuning and overfitting monitoring, and the test set is used for final model performance evaluation. After training, a trained model is obtained, which has the ability to map output labels from input features. This trained model serves as a priori model of real traffic participant behavior.

[0129] This embodiment uses graph neural networks to explicitly model the spatial interaction relationships between traffic participants. Compared with traditional sequence models, it can more accurately capture the spatial dependencies and game relationships in multi-vehicle interaction scenarios, thereby generating more realistic motion response behaviors.

[0130] In some embodiments of the present invention, behavioral reasoning processing is performed on the driving behavior currently executed by the simulated vehicle and the real-time traffic status fed back by the simulation platform to obtain motion response commands for multiple traffic participants, and the motion status of multiple traffic participants is updated according to the motion response commands. Specifically, the following processing steps are executed cyclically within each simulation step: First, the latest driving behavior commands executed by the simulated vehicle, as well as the current motion state and relative positional relationship of each individual among multiple traffic participants, are processed to obtain the model input features.

[0131] As can be understood, the simulation step size refers to the time interval between two consecutive state updates in the simulation platform. The selection of the simulation step size needs to strike a balance between simulation accuracy and computational load; typical values ​​include 0.05 seconds and 0.1 seconds, corresponding to simulation frequencies of 20Hz and 10Hz, respectively. At the beginning of each simulation step, the driving behavior command of the simulated vehicle at the current moment is first read from the simulation platform's state interface. This command consists of the steering wheel angle, throttle opening, and braking pressure values ​​output by the autonomous driving model in the previous simulation step and processed by command calculation. Simultaneously, the current motion state of all other traffic participants in the simulation platform is read, including their position coordinates, velocity vector, acceleration vector, and heading angle. Then, the relative positional relationship between each traffic participant and the simulated vehicle is calculated, including longitudinal and lateral distances. The longitudinal distance is defined as the directed distance from the center of the simulated vehicle's rear axle along the vehicle's heading direction to the position of the traffic participant; the lateral distance is defined as the directed distance from the center of the simulated vehicle's rear axle along the direction perpendicular to the vehicle's heading direction to the position of the traffic participant.

[0132] Data assembly processing refers to organizing and stitching together the scattered data according to the format and dimensions required by the model input. Specific operations include converting physical quantities from the simulation engine's internal unit system to the model's desired unit system; for example, converting velocity from meters per second to kilometers per hour, and acceleration from meters per second squared to g. It also includes data normalization, mapping feature components with different dimensions and value ranges to a unified numerical interval. Common normalization methods are minimum-maximum normalization (MINMR) or z-score normalization. MINMR normalization linearly maps the original data to the interval 0 to 1. z-score normalization subtracts the mean from the original data and divides by the standard deviation to make it follow a standard distribution with a mean of 0 and a standard deviation of 1. The normalized feature components are then stitched together in a fixed order to form a one-dimensional feature vector, which serves as the model input feature.

[0133] Then, the model input features are fed into the prior model of real traffic participant behavior for forward inference processing to obtain the acceleration and steering angle commands of multiple traffic participants in the next simulation step, which serve as motion response commands.

[0134] As can be understood, forward inference refers to the process of feeding the assembled model's input feature vectors into the trained graph neural network, performing message passing and node feature update calculations at each layer, and finally obtaining the predicted acceleration and steering angle values ​​for each traffic participant node at the output layer. Backpropagation and parameter updates are not performed during inference; only forward computation is executed. Inference computation can be performed on a central processing unit (CPU) or a graphics processing unit (GPU). A GPU is a processor with a large number of parallel computing cores, making it particularly suitable for computationally intensive tasks such as matrix multiplication in neural networks. In simulation evaluation scenarios requiring real-time execution, GPUs are typically used for inference to ensure real-time performance.

[0135] Finally, the motion response commands are processed to obtain motion state update parameters, and the motion states of multiple traffic participants in the simulation platform are updated based on the motion state update parameters.

[0136] Command processing refers to inputting the acceleration and steering angle commands output by the model into the vehicle kinematics or dynamics update module to calculate the motion state update parameters of each traffic participant in the next simulation step. If a kinematic model is used, the velocity and displacement at the next moment are calculated using the uniform acceleration motion formula based on the current velocity, acceleration command, and simulation step size. The change in heading angle is then calculated using a bicycle model in conjunction with the steering angle command and wheelbase.

[0137] The bicycle model simplifies a four-wheeled vehicle into a two-wheeled bicycle kinematic model, with the core assumption that the left and right wheels are combined into a single wheel and move only in a plane. A dynamic model further considers factors such as tire-ground friction, vehicle mass distribution, and suspension characteristics, obtaining more accurate state updates through numerically solving the differential equations of motion. The motion state update parameters include new position coordinates, velocity vector, acceleration vector, and heading angle.

[0138] Updating the motion state of multiple traffic participants in the simulation platform based on motion state update parameters refers to calling the simulation platform's state setting interface to update the motion state of each traffic participant from its current value to the calculated new value. Upon receiving the state update, the simulation platform synchronously refreshes the 3D rendering screen, so that the traffic participants visually exhibit corresponding displacement and posture changes. This completes the behavior inference and state update loop within one simulation step.

[0139] This embodiment continuously executes a loop of data assembly, model inference, and state updates within each simulation step, enabling other traffic participants in the simulation environment to make continuous and dynamic behavioral responses to the driving behavior of the simulated vehicle, thus forming a realistic closed-loop interactive simulation environment.

[0140] In some embodiments of the present invention, the driving performance of the autonomous driving model is evaluated in a multi-dimensional manner based on multiple evaluation dimensions to obtain the evaluation results of the autonomous driving model. Specifically, the evaluation includes the following six processing steps: Firstly, the ratio of recorded collision counts to total mileage is calculated to obtain the collision rate, which characterizes safety. The number of collisions refers to the total number of events in which the simulated vehicle comes into contact with other traffic participants or static obstacles in the scene during the entire simulation evaluation. The criterion for determining a collision event is the spatial overlap between the simulated vehicle's collision detection bounding box and the collision detection bounding boxes of other objects in the simulation engine. A collision detection bounding box is a simplified geometric shape that surrounds the 3D model, used for efficient collision detection calculations; it is typically a cuboid or cylinder. Total mileage refers to the cumulative distance traveled by the simulated vehicle from the starting point to the end point during the entire evaluation process, measured in kilometers. The ratio calculation involves dividing the number of collisions by the total mileage to obtain the number of collisions per unit mileage, i.e., the collision rate. The lower the value, the higher the safety of the autonomous driving model.

[0141] Secondly, the ratio of the recorded number of traffic light violations to the total number of traffic light encounters is calculated to obtain the traffic rule compliance rate, which characterizes compliance. A traffic light violation refers to the simulated vehicle's failure to stop before the stop line as required by traffic regulations when the traffic light is in the red phase. The stop line is a horizontal marking at the intersection entrance lane that vehicles are prohibited from crossing during a red light. The total number of traffic light encounters refers to the sum of all traffic light-controlled intersection entrance lanes traversed by the simulated vehicle during the evaluation process. The ratio calculation involves dividing the number of violations by the total number of encounters and then multiplying by 100% to obtain the traffic rule compliance rate expressed as a percentage. The closer the compliance rate is to 100%, the higher the level of traffic rule compliance of the model.

[0142] Thirdly, statistical analysis is performed on the number of times the autonomous driving model calls the roadside view data stream in the vehicle-end perception blind spot scenario to obtain the collaborative perception utilization rate used to characterize the collaboration. The vehicle-end perception blind spot refers to the spatial area that the on-board virtual sensor of the simulation vehicle cannot observe due to building obstruction, large vehicle obstruction, or the limitation of the sensor's own field of view.

[0143] During simulation, whether a target object is in a blind spot is determined by whether it is located inside the field of view cone of the onboard sensor. The field of view cone refers to the conical area in space defined by the field of view angle starting from the sensor position. Calling the roadside view data stream means that the autonomous driving model actually reads and processes the perception information in the roadside view data stream within its perception or decision-making module to compensate for the missing environmental information in the vehicle's perception blind spot. The method for determining whether the model calls roadside data is to set monitoring points at key processing nodes within the model to record the time when roadside data is accessed and the corresponding scene events. Statistical analysis processing refers to the ratio of the number of times the model actually calls roadside data to assist decision-making when encountering a vehicle-to-everything (V2X) scenario to the total number of V2X scenarios encountered. This ratio is the collaborative perception utilization rate. A higher collaborative perception utilization rate indicates that the model makes fuller use of vehicle-road cooperative information.

[0144] Fourthly, the fluctuation amplitude of lateral and longitudinal acceleration during driving is calculated to obtain an average comfort score characterizing comfort. Lateral acceleration refers to the acceleration experienced by the vehicle in the lateral direction perpendicular to the driving direction, generated during lateral movements such as turning and lane changing, and is measured in meters per second squared. Longitudinal acceleration refers to the acceleration of the vehicle along the driving direction, generated during acceleration and braking. The fluctuation amplitude calculation involves calculating the standard deviation or peak-to-valley difference of the lateral and longitudinal acceleration time-series signals recorded throughout the evaluation process within a series of fixed time windows, and then averaging the standard deviations of all windows to obtain the average fluctuation amplitude of lateral and longitudinal acceleration. The standard deviation is the square root of the sum of the squares of the differences between each data point and the mean, used to measure the dispersion of the data. The peak-to-valley difference is the difference between the maximum and minimum values ​​of the signal within a time window. The average fluctuation amplitudes of lateral and longitudinal acceleration are weighted and combined according to preset weights, mapped to a comfort score interval to obtain an average comfort score. A higher score indicates better ride comfort.

[0145] Fifthly, the duration of the unattended driving task is evaluated to obtain a long-term decision robustness score, which characterizes robustness. Unattended driving means that during the simulation evaluation, the autonomous driving model independently completes all driving tasks without requiring human safety intervention. Triggering conditions requiring human safety intervention include a collision, leaving the drivable area boundary, violating traffic signals with safety risks, or prolonged stagnation. The drivable area boundary refers to the spatial boundary of the vehicle's passable area defined by the road edge or curb. A continuous driving task refers to the entire driving process from the start of the simulation until the triggering of the intervention condition or the completion of the preset evaluation time. The evaluation process involves recording the duration of unattended continuous driving, dividing the actual unattended time by the preset total evaluation time, and then multiplying by 100% to obtain a completion rate expressed as a percentage. This completion rate serves as the long-term decision robustness score. A higher score indicates stronger decision stability and robustness of the model during long-term continuous driving.

[0146] The sixth aspect involves a comprehensive analysis of at least two of the following: collision rate, traffic rule compliance rate, collaborative perception utilization rate, average comfort score, and long-term decision robustness score, to obtain the evaluation results.

[0147] One approach to comprehensive analysis and processing is the weighted summation method: Each of the five evaluation indicators is assigned a weight coefficient, and the sum of all weight coefficients is 1. Each indicator value is multiplied by its weight coefficient, and the results are summed to obtain the overall score. The allocation of weight coefficients depends on the specific evaluation objectives and focus. For example, if the evaluation focus is on safety verification, a higher weight is assigned to the collision rate; if the evaluation focus is on collaborative capability verification, a higher weight is assigned to the collaborative perception utilization rate.

[0148] Another approach to comprehensive analysis and processing is the graded evaluation method: A threshold is pre-set for each evaluation indicator, and the success rate of each indicator in reaching the corresponding threshold is statistically analyzed. Based on the number of indicators meeting the threshold or the success rate of a specific combination of indicators, a graded evaluation conclusion is given, such as four levels: excellent, good, satisfactory, and unsatisfactory. The evaluation results are output in the form of a structured data file or evaluation report, including the individual values ​​of each indicator, the overall score or graded conclusion, and a list of key events and timestamps during the evaluation process.

[0149] This embodiment uses multi-dimensional quantitative evaluation indicators and comprehensive analysis methods to comprehensively evaluate the performance of autonomous driving models from multiple perspectives, including safety, compliance, collaboration, comfort, and robustness.

[0150] In some embodiments of the present invention, the simulation platform based on multi-source data packets further includes the following processing steps: First, the traffic signal status data in the multi-source data packets is processed by signal timing analysis to obtain the traffic signal control timing.

[0151] It is understandable that traffic signal status data refers to time-series data recorded during the data collection process at real traffic intersections, showing the signal phase state of traffic lights at various times. A signal phase refers to the right-of-way state assigned by a traffic light to a specific group of traffic flows. A complete signal cycle contains multiple signal phases that appear sequentially in a fixed order.

[0152] Common signal phases include: north-south straight-ahead phase, north-south left-turn phase, east-west straight-ahead phase, and east-west left-turn phase. Within each phase cycle, traffic lights sequentially experience three light states: green, yellow, and red. A green light indicates that vehicles in that direction are permitted to proceed; a yellow light indicates that the green light is about to end, reminding vehicles to slow down and stop; and a red light indicates that vehicles in that direction are prohibited from proceeding. Signal timing analysis processing involves extracting the start and end times of each signal light group's phase and its light state information within each signal cycle from the traffic signal state data, and organizing this information into a timing table data structure. Each row of the timing table records the start time, end time, and light type of a light state segment. The light type is represented by an enumeration value, such as green corresponding to green light, yellow corresponding to yellow light, and red corresponding to red light. The timing table is arranged in ascending order of start time, forming the traffic signal control timing sequence.

[0153] Then, the traffic signal control timing is aligned with the time base of the dynamic traffic flow to obtain the aligned traffic signal control timing.

[0154] Understandably, the time reference for dynamic traffic flow refers to the origin of the time coordinates used by the trajectory data of all traffic participants in the dynamic traffic flow, which is usually consistent with the time reference uniformly used in multi-source data packets. Alignment processing involves checking whether the time values ​​in the traffic signal control timing data and the timestamps in the dynamic traffic flow trajectory data are under the same time reference. If a time offset exists, compensation and adjustment are performed to ensure that the switching times of traffic lights are precisely synchronized with the movement times of vehicles in the traffic flow. Time offsets may originate from clock deviations between the signal status data acquisition equipment and the trajectory data acquisition equipment. This deviation has already been eliminated in the spatiotemporal alignment processing stage. This step of alignment processing mainly verifies the parsed timing data and performs time shift corrections when necessary.

[0155] Finally, based on the aligned traffic signal control timing, the virtual traffic lights in the simulation platform are driven to switch states.

[0156] As can be understood, a virtual traffic light refers to a 3D model of a traffic light with a programmable control interface, set up in a static digital twin scenario. The driving operation refers to the simulation platform reading the aligned traffic signal control timing sequence within each simulation step, looking up the corresponding light color status in the timing table based on the current simulation time, and then calling the virtual traffic light's status setting interface to update its display status to the corresponding light color. The status setting interface typically supports directly setting the light color status enumeration value, or setting the on / off Boolean values ​​for the red, yellow, and green lights in each light group. A Boolean value is a logical data type that takes only two values: true or false. True indicates the corresponding light color is on, and false indicates the corresponding light color is off. Through the above driving operation, the virtual traffic lights in the simulation platform can strictly switch light colors according to the traffic signal control timing sequence collected from the real intersection, making the traffic signal control in the simulation environment highly consistent with the real intersection.

[0157] This embodiment analyzes real traffic signal state data and synchronously drives virtual traffic lights in the simulation platform, making the traffic signal control rules in the simulation evaluation scenario completely consistent with the real traffic environment, thereby further improving the authenticity and reliability of the simulation evaluation results.

[0158] The simulation evaluation device for vehicle-road cooperative autonomous driving model provided by the present invention will be described below. The simulation evaluation device for vehicle-road cooperative autonomous driving model described below can be referred to in correspondence with the simulation evaluation method for vehicle-road cooperative autonomous driving model described above, and will not be repeated here.

[0159] like Figure 3 The diagram shown is a structural schematic of a simulation evaluation device for a vehicle-road cooperative autonomous driving model provided by the present invention. The simulation evaluation device for a vehicle-road cooperative autonomous driving model includes the following modules: The acquisition module 310 is used to acquire multi-source data packets from real traffic scenarios and configure the simulation platform based on the multi-source data packets; The injection module 320 is used to inject vehicle-road cooperative perception information into the autonomous driving model under test when the autonomous driving model is connected to the configured simulation platform to obtain driving behavior instructions; wherein, the driving behavior instructions are used to drive the movement of the simulation vehicle in the simulation platform, the simulation vehicle is configured with the autonomous driving model, and the vehicle-road cooperative perception information is obtained based on the multi-source data packet; The reasoning module 330 is used to perform behavioral reasoning processing on the driving behavior currently executed by the simulated vehicle and the real-time traffic status fed back by the simulation platform based on a preset prior model of the behavior of real traffic participants, to obtain motion response commands of multiple traffic participants, and to update the motion status of the multiple traffic participants according to the motion response commands. The evaluation module 340 is used to perform multi-dimensional evaluation of the driving performance of the autonomous driving model based on multiple evaluation dimensions when the simulated vehicle interacts with the multiple traffic participants for a preset time, so as to obtain the evaluation result of the autonomous driving model.

[0160] Figure 4 An example is a schematic diagram of the physical structure of an electronic device, such as... Figure 4 As shown, the electronic device may include: a processor 410, a communications interface 420, a memory 430, and a communications bus 440, wherein the processor 410, the communications interface 420, and the memory 430 communicate with each other through the communications bus 440. The processor 410 can call logic instructions in the memory 430 to execute a simulation evaluation method for a vehicle-road cooperative autonomous driving model. This method includes: acquiring multi-source data packets from a real traffic scenario and configuring a simulation platform based on the multi-source data packets; when the autonomous driving model to be tested is connected to the configured simulation platform, injecting vehicle-road cooperative perception information into the autonomous driving model to obtain driving behavior instructions; wherein the driving behavior instructions are used to drive the movement of the simulated vehicle in the simulation platform, the simulated vehicle is configured with the autonomous driving model, and the vehicle-road cooperative perception information is obtained based on the multi-source data packets; based on a preset real traffic participant behavior prior model, performing behavioral reasoning processing on the driving behavior currently executed by the simulated vehicle and the real-time traffic state fed back by the simulation platform to obtain motion response instructions for multiple traffic participants, and updating the motion state of the multiple traffic participants according to the motion response instructions; and, under the condition that the simulated vehicle continuously interacts with the multiple traffic participants for a preset duration, performing multi-dimensional evaluation processing on the driving performance of the autonomous driving model based on multiple evaluation dimensions to obtain the evaluation result of the autonomous driving model.

[0161] Furthermore, the logical instructions in the aforementioned memory 430 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0162] On the other hand, the present invention also provides a computer program product, which includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the simulation evaluation method for vehicle-road cooperative autonomous driving models provided by the above methods. The method includes: acquiring multi-source data packets from real traffic scenarios and configuring a simulation platform based on the multi-source data packets; when the autonomous driving model to be tested is connected to the configured simulation platform, injecting vehicle-road cooperative perception information into the autonomous driving model to obtain driving behavior instructions; wherein the driving behavior instructions are used to drive the simulation platform. The simulation vehicle moves within the system, and the simulation vehicle is equipped with the autonomous driving model. The vehicle-road cooperative perception information is obtained based on the multi-source data packets. Based on a preset prior model of real traffic participant behavior, behavioral reasoning processing is performed on the driving behavior currently executed by the simulation vehicle and the real-time traffic status fed back by the simulation platform to obtain motion response commands for multiple traffic participants. The motion status of the multiple traffic participants is updated according to the motion response commands. Under the condition that the simulation vehicle and the multiple traffic participants continuously interact for a preset time, the driving performance of the autonomous driving model is evaluated in a multi-dimensional manner based on multiple evaluation dimensions to obtain the evaluation result of the autonomous driving model.

[0163] In another aspect, the present invention also provides a non-transitory computer-readable storage medium storing a computer program thereon. When executed by a processor, the computer program implements a simulation evaluation method for a vehicle-road cooperative autonomous driving model provided by the above methods. The method includes: acquiring multi-source data packets of a real traffic scenario and configuring a simulation platform based on the multi-source data packets; when the autonomous driving model to be tested is connected to the configured simulation platform, injecting vehicle-road cooperative perception information into the autonomous driving model to obtain driving behavior instructions; wherein the driving behavior instructions are used to drive the movement of a simulated vehicle in the simulation platform, the simulated vehicle is configured with the autonomous driving model, and the vehicle-road cooperative perception information is obtained based on the multi-source data packets; based on a preset real traffic participant behavior prior model, performing behavioral reasoning processing on the driving behavior currently executed by the simulated vehicle and the real-time traffic state fed back by the simulation platform to obtain motion response instructions for multiple traffic participants, and updating the motion state of the multiple traffic participants according to the motion response instructions; when the simulated vehicle and the multiple traffic participants continuously interact for a preset duration, performing multi-dimensional evaluation processing on the driving performance of the autonomous driving model based on multiple evaluation dimensions to obtain the evaluation result of the autonomous driving model.

[0164] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0165] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0166] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A simulation evaluation method for vehicle-road cooperative autonomous driving models, characterized in that, include: Acquire multi-source data packets from real traffic scenarios and configure a simulation platform based on the multi-source data packets; When the autonomous driving model to be tested is connected to the configured simulation platform, vehicle-road cooperative perception information is injected into the autonomous driving model to obtain driving behavior instructions; wherein, the driving behavior instructions are used to drive the movement of the simulated vehicle in the simulation platform, the simulated vehicle is configured with the autonomous driving model, and the vehicle-road cooperative perception information is obtained based on the multi-source data packets; Based on a pre-defined prior model of real traffic participant behavior, behavioral reasoning is performed on the driving behavior currently executed by the simulated vehicle and the real-time traffic status fed back by the simulation platform to obtain motion response commands for multiple traffic participants, and the motion status of the multiple traffic participants is updated according to the motion response commands. Under the condition that the simulated vehicle interacts with the multiple traffic participants for a preset duration, the driving performance of the autonomous driving model is evaluated in a multi-dimensional manner based on multiple evaluation dimensions to obtain the evaluation result of the autonomous driving model.

2. The simulation evaluation method for vehicle-road cooperative autonomous driving models according to claim 1, characterized in that, The acquisition of multi-source data packets from real traffic scenarios includes: Spatiotemporal alignment processing is performed on road structure data, traffic participant trajectory data, traffic signal status data, vehicle sensor data, and roadside perception data that are collected synchronously at real traffic intersections based on the same time reference to obtain spatiotemporal synchronized data. The multi-source data packet is obtained by associating and labeling the sensor data of the same traffic participant under different sensors in the spatiotemporal synchronization data.

3. The simulation evaluation method for vehicle-road cooperative autonomous driving models according to claim 2, characterized in that, The simulation platform configured based on the multi-source data packets includes: The road structure data in the multi-source data packets are processed by automated road network generation to obtain an initial digital twin static scene; The initial digital twin static scene is manually modified to obtain a modified digital twin static scene, and the modified digital twin static scene is loaded into the simulation platform.

4. The simulation evaluation method for vehicle-road cooperative autonomous driving models according to claim 2, characterized in that, The simulation platform configured based on the multi-source data packets includes: The traffic participant trajectory data in the multi-source data packet is processed by real trajectory playback to obtain the first type of traffic flow segment; The trajectory data of the traffic participants are processed by data-driven trajectory generation to obtain the second type of traffic flow segment; The first type of traffic flow segment and the second type of traffic flow segment are subjected to spatiotemporal fusion processing to obtain a dynamic traffic flow composed of the multiple traffic participants; Based on the dynamic traffic flow, the simulation platform is configured with traffic participant instances and environmental parameters. The configuration process includes at least one of the following: parsing the type identifier, geometric dimensions, appearance model identifier, and initial pose contained in the trajectory data of the traffic participants to obtain instance attribute configuration parameters, and generating three-dimensional solid models of the multiple traffic participants in the simulation platform according to the instance attribute configuration parameters; and parsing the weather type, light intensity value, and time period information recorded in the trajectory data of the traffic participants to obtain environmental simulation parameters, and configuring the weather particle system and light source parameters of the simulation platform according to the environmental simulation parameters.

5. The simulation evaluation method for vehicle-road cooperative autonomous driving models according to claim 4, characterized in that, Before the step of configuring traffic participant instances and environmental parameters on the simulation platform based on the dynamic traffic flow, the method includes: Spatiotemporal overlap detection processing is performed on the first type of traffic flow segment and the second type of traffic flow segment to identify conflict trajectory segments with spatiotemporal conflicts; The conflict trajectory segments are processed to resolve conflicts, resulting in the dynamic traffic flow free of spatiotemporal conflicts.

6. The simulation evaluation method for vehicle-road cooperative autonomous driving models according to claim 5, characterized in that, The process of resolving conflicts in the conflict trajectory segments to obtain the dynamic traffic flow free of spatiotemporal conflicts includes: The velocity difference sequence is obtained by performing difference calculation on the velocity sequences of adjacent trajectories in the conflict trajectory segment; The velocity difference sequence is smoothed by filtering to obtain a smoothed velocity sequence. Based on the smoothed velocity sequence, the conflict trajectory segment is resampled to obtain a smoothed trajectory segment. For the trajectories that still have spatiotemporal conflicts after trajectory resampling in the conflict trajectory segments, local path replanning is performed to obtain the replanned trajectory segments. The trajectory segments after smooth transition and the trajectory segments after replanning are spliced ​​together to obtain the dynamic traffic flow without spatiotemporal conflicts.

7. The simulation evaluation method for vehicle-road cooperative autonomous driving models according to claim 2, characterized in that, The vehicle-road cooperative perception information includes at least: vehicle-side view data stream and roadside view data stream; The vehicle-road cooperative perception information is obtained based on the multi-source data packets, including: Based on the vehicle sensor data in the multi-source data packet, sensor configuration processing is performed on the virtual vehicle sensor to obtain the configured virtual vehicle sensor. The configured virtual vehicle sensors are processed to generate data, resulting in a vehicle-side view data stream. Based on the roadside sensing data in the multi-source data packet, the virtual roadside sensor is configured to obtain the configured virtual roadside sensor. The configured virtual roadside sensors are processed to generate a roadside view data stream.

8. The simulation evaluation method for vehicle-road cooperative autonomous driving models according to claim 7, characterized in that, The process of injecting vehicle-road cooperative perception information into the autonomous driving model to obtain driving behavior instructions includes: The vehicle-side view data stream and the roadside view data stream are time-stamped and synchronized to obtain a synchronized vehicle-road cooperative data stream. The synchronized vehicle-road cooperative data stream is encapsulated to obtain a message queue that conforms to a preset vehicle-road cooperative communication protocol. The message queue is sent to the autonomous driving model in real time to complete the injection of the vehicle-road cooperative perception information; The autonomous driving model performs instruction processing on the control instructions output by the message queue to obtain the driving behavior instructions used to drive the simulated vehicle.

9. The simulation evaluation method for vehicle-road cooperative autonomous driving models according to claim 2, characterized in that, The training steps for the prior model of real traffic participant behavior include: The interaction trajectory segments between the multiple traffic participants extracted from the multi-source data packets are processed to form a sample dataset. Each sample uses the driving behavior of the simulated vehicle at consecutive time points, the motion state of each individual among the multiple traffic participants, and the relative positional relationship between each individual among the multiple traffic participants and the simulated vehicle as input features, and the acceleration command and steering angle command of each individual among the multiple traffic participants at the next time point as output labels. Using the sample dataset, a preset graph neural network is trained to obtain a trained model; wherein, the trained model has the ability to obtain the output label based on the input feature mapping, and the trained model serves as a prior model of the behavior of the real traffic participants.

10. A simulation and evaluation device for vehicle-road cooperative autonomous driving models, characterized in that, include: The acquisition module is used to acquire multi-source data packets from real traffic scenarios and configure the simulation platform based on the multi-source data packets; The injection module is used to inject vehicle-road cooperative perception information into the autonomous driving model under test when the autonomous driving model is connected to the configured simulation platform to obtain driving behavior instructions; wherein, the driving behavior instructions are used to drive the movement of the simulated vehicle in the simulation platform, the simulated vehicle is configured with the autonomous driving model, and the vehicle-road cooperative perception information is obtained based on the multi-source data packet; The reasoning module is used to perform behavioral reasoning processing on the driving behavior currently performed by the simulated vehicle and the real-time traffic status fed back by the simulation platform based on a preset prior model of the behavior of real traffic participants, to obtain motion response commands of multiple traffic participants, and to update the motion status of the multiple traffic participants according to the motion response commands. The evaluation module is used to perform multi-dimensional evaluation of the driving performance of the autonomous driving model based on multiple evaluation dimensions, under the condition that the simulated vehicle interacts with the multiple traffic participants for a preset time, and obtain the evaluation result of the autonomous driving model.

Images