An autonomous driving closed-loop simulation and multi-dimensional online evaluation method and system based on an asynchronous architecture

Abstract

The application discloses a kind of based on asynchronous architecture's automatic driving closed loop simulation and multidimensional online evaluation method and system. Including S1, establish asynchronous closed loop interactive environment and initialization setting;S2, configure the asynchronous operation pipeline of vehicle intelligent agent, make simulation process with asynchronous multi-process mode collaborative operation;S3, establish by simulation server or interactive environment driven state subscription mechanism, asynchronously obtain the required each real-time state;S4, start independently running asynchronous multidimensional online evaluation system, based on real-time state calculation safety, efficiency, comfort and interaction harmony degree evaluation index;S5, introduce execution delay semantic monitoring mechanism, and judge whether the automatic driving algorithm to be evaluated is mapped to closed loop dynamics;S6, record various data in long-range closed loop interaction process, generate multidimensional automatic driving performance evaluation report after evaluation ends.The application overcomes the time freezing bottleneck of traditional synchronous simulation, and provides comprehensive and objective multidimensional online evaluation system.

Description

Technical Field

[0001] This invention relates to the fields of autonomous driving simulation testing, closed-loop interactive evaluation, asynchronous computing architecture, intelligent vehicle performance evaluation, and autonomous driving algorithm verification, and particularly to an autonomous driving closed-loop simulation and multi-dimensional online evaluation method and system based on an asynchronous architecture. Background Technology

[0002] Before being deployed on real roads, autonomous driving systems typically require large-scale simulation testing to verify their perception, prediction, planning, control, and end-to-end decision-making capabilities. Closed-loop simulation, as an important means of verifying autonomous driving algorithms, enables these algorithms to generate driving decisions in real time based on sensor inputs in a simulated environment, and influence subsequent environmental states through vehicle dynamics feedback, thus forming a continuous interactive process.

[0003] Existing autonomous driving simulation testing platforms typically employ a synchronous simulation mechanism. Under this mechanism, the progress of the simulated world often requires waiting for the vehicle's intelligent agent to complete computational steps such as sensor data acquisition, data processing, algorithm inference, and vehicle control. While this mode is beneficial for achieving frame-level synchronization and result reproduction, it also introduces significant execution semantic biases: when the autonomous driving algorithm being evaluated has a long computation time, the simulated world may be in a paused or waiting state, causing the actual physical computation delay of the algorithm to fail to be reflected in the vehicle's closed-loop behavior. Consequently, the test results obtained are difficult to accurately reflect the response latency, control lag, and risks caused by computational time in a real-world deployment environment.

[0004] On the other hand, existing autonomous driving evaluation systems typically focus on offline statistical indicators such as task completion, collisions, and route deviations, or rely on a unified scoring mechanism after the task ends. For factors such as continuous changes in algorithm behavior during long-range closed-loop interactions, the degree to which background traffic flow is affected, changes in vehicle comfort, compliance with traffic rules, and execution latency semantics, existing methods lack a unified online evaluation framework. Especially in open traffic flow environments or complex interaction scenarios, relying solely on task-end metrics is insufficient to comprehensively evaluate the overall performance of autonomous driving algorithms in terms of safety, efficiency, comfort, and interaction harmony.

[0005] Furthermore, in autonomous driving simulation testing, data such as background vehicles, pedestrians, traffic lights, stop signs, road topology, route progress, and vehicle control status are characterized by multi-source heterogeneity, temporal dynamics, and high-frequency updates. If the evaluation process is strongly coupled with the advancement of the simulation world or the vehicle agent's computation process, the evaluation computation itself may further increase simulation obstruction, affecting the realism and continuity of the test results. Therefore, a method is needed that can collect environmental states in real time and perform multi-dimensional online evaluation without obstructing the advancement of the simulation world or interfering with the closed-loop control of the vehicle agent.

[0006] To address the aforementioned issues, this invention proposes an asynchronous architecture-based method and system for closed-loop simulation and multi-dimensional online evaluation of autonomous driving. By decoupling the simulation world, the vehicle intelligent agent's operational pipeline, and the background evaluation system through an asynchronous multi-process approach, the actual computational latency of the autonomous driving algorithm can be mapped to the closed-loop dynamics. Furthermore, it continuously outputs multi-dimensional evaluation results during long-term interaction, thereby improving the realism, continuity, and evaluation completeness of autonomous driving simulation testing. Summary of the Invention

[0007] The purpose of this invention is to propose a method and system for closed-loop simulation and multi-dimensional online evaluation of autonomous driving based on an asynchronous architecture. First, an asynchronous closed-loop interactive environment for autonomous driving is established, initializing the simulation world, vehicle agents, test routes, background traffic participants, and system communication interfaces, and setting the simulation world to advance with continuous-time semantics. Second, an asynchronous execution pipeline for the vehicle agents is configured, enabling sensor data acquisition, data preprocessing, autonomous driving algorithm execution, and underlying vehicle control to operate collaboratively in an asynchronous multi-process manner without blocking the simulation world's progress. Then, an event-driven real-time state acquisition mechanism is run to asynchronously acquire real-time state snapshots of the vehicle agents, background traffic participants, traffic signals, and road topology. Next, an asynchronous multi-dimensional online evaluation system is started, running an independent background evaluation process to calculate safety, efficiency, comfort, and interaction harmony evaluation indicators based on the real-time state snapshots. Furthermore, an execution latency semantic monitoring mechanism is introduced to record end-to-end physical wall clock latency and simulation elapsed latency, and calculate the latency consistency ratio to determine whether the computational latency of the autonomous driving algorithm to be evaluated is mapped into the closed-loop dynamics. Finally, during the long-range closed-loop interaction process, the evaluation indicators of each dimension, the latency consistency ratio, specific violations and background traffic flow time series are continuously recorded, and a multi-dimensional autonomous driving performance evaluation report is generated after the evaluation is completed.

[0008] To achieve the above objectives, the technical solution provided by this invention is as follows:

[0009] A closed-loop simulation and multi-dimensional online evaluation method for autonomous driving based on an asynchronous architecture, the method comprising:

[0010] S1. Establish an asynchronous closed-loop interactive environment for autonomous driving, initialize the simulation world, vehicle intelligent agent, test route, background traffic participants and system communication interface, and set the simulation world to advance with continuous time semantics.

[0011] S2. Configure an asynchronous operation pipeline for the vehicle intelligent agent, so that sensor data acquisition, data preprocessing, autonomous driving algorithm execution and underlying vehicle control can run in a collaborative manner in an asynchronous multi-process manner without blocking the progress of the simulation world.

[0012] S3. Run an event-driven real-time state acquisition mechanism, establish a state subscription mechanism driven by a simulation server or interactive environment, and asynchronously acquire real-time state snapshots of vehicle intelligent agents, background traffic participants, traffic signals and road topology.

[0013] S4. Start the asynchronous multidimensional online evaluation system, run an independent background evaluation process, and calculate the evaluation indicators of safety, efficiency, comfort and interaction harmony based on real-time status snapshots.

[0014] S5. Introduce an execution latency semantic monitoring mechanism to record end-to-end physical wall clock latency and simulation elapsed latency, and calculate the latency consistency ratio to determine whether the computation latency of the autonomous driving algorithm to be evaluated is mapped to the closed-loop dynamics.

[0015] S6. During the long-range closed-loop interaction process, continuously record multi-dimensional evaluation data, violation event data, background traffic flow data, and time delay semantic data, and generate a multi-dimensional autonomous driving performance evaluation report after the evaluation is completed.

[0016] To further explain, step S1 specifically includes:

[0017] S11. Start the autonomous driving closed-loop simulation system, load the autonomous driving simulation world, and initialize the test map, test route, traffic signals, road topology, background traffic participants, the main control vehicle corresponding to the vehicle intelligent agent, and system operating parameters.

[0018] S12. Create at least one vehicle intelligent agent and configure the vehicle intelligent agent with a sensor kit, vehicle status interface, navigation interface, control interface, autonomous driving algorithm interface and evaluation data interface;

[0019] S13. Establish a communication connection with the simulation server or interactive environment, and obtain interface objects for simulation world state query, map information query, vehicle generation, traffic participant management, sensor management and event listening.

[0020] S14. Set the simulation environment stepping mode to asynchronous mode and configure the minimum stepping interval so that the simulation world advances with continuous time semantics, and its time advancement does not need to wait for the vehicle intelligent agent to complete sensor data acquisition, data preprocessing, autonomous driving algorithm execution or underlying vehicle control.

[0021] S15. The simulation world is deployed on the first computing node, and the asynchronous operation pipeline of the vehicle intelligent agent is deployed on the second computing node. The first computing node and the second computing node interact with each other through Ethernet, LAN or wireless network communication links, so that the simulation world advancement task and the vehicle intelligent agent computing task are executed in parallel on different computing nodes, decoupling the simulation load and the computing load, and reducing the impact of the vehicle intelligent agent's computing time on the continuous time advancement of the simulation world.

[0022] S16. Initialize the unified signal system, which is used to broadcast global control signals to the sensor data acquisition process, data preprocessing process, autonomous driving algorithm execution process, underlying vehicle control process and background evaluation process when the scene is initialized, the scene is reset, the simulation starts, the simulation is paused, the abnormal interruption occurs, the evaluation ends and the system is terminated.

[0023] S17. Configure preset routes, loop routes, behavioral strategies or traffic management strategies for background traffic participants, and set discrete route checkpoints for background traffic participants so that background traffic participants can maintain local interaction degrees of freedom while maintaining the consistency of macro traffic flow structure in repeated tests.

[0024] S18. Initialize the data structure for storing evaluation data. The data structure includes a master vehicle indicator data structure for storing the state and evaluation indicators of the vehicle intelligent agent, a global traffic indicator data structure for storing the state of background traffic participants and traffic flow time series, and a violation log data structure for recording specific violation events.

[0025] To further explain, step S2 specifically involves:

[0026] S21. Start the sensor data acquisition process for the vehicle intelligent agent. The sensor data acquisition process asynchronously acquires multimodal sensor data according to the sampling frequency of each sensor. The data includes at least one of the following: front view image, multi-view image, lidar point cloud, millimeter-wave radar data, inertial measurement data, global navigation satellite system data, vehicle speed, vehicle attitude, chassis status, navigation commands, and historical trajectory.

[0027] S22. The sensor data acquisition process continuously writes sensor data through callback mechanism, publish-subscribe mechanism, shared cache, shared memory or message queue. The data generation process of different sensors does not need to be completed synchronously in the same simulation frame or the same physical moment.

[0028] S23. Initiate a data preprocessing process for the vehicle intelligent agent. This data preprocessing process, acting as an asynchronous consumer of sensor data, continuously reads the latest available multimodal sensor data and performs timestamp calibration, multi-source data alignment, coordinate system transformation, normalization, image preprocessing, point cloud preprocessing, vehicle state encoding, and navigation command encoding on the data to generate a unified input representation. This unified input representation includes image tensors, point cloud tensors, bird's-eye view features, vehicle state vectors, navigation commands, historical motion states, or combinations thereof, used to mask differences between different sensor input formats.

[0029] S24. The data preprocessing process writes the unified input representation into shared memory, cache queue, publish-subscribe channel or inter-process communication channel for the autonomous driving algorithm execution process to read.

[0030] S25. The vehicle intelligent agent initiates the execution process of the autonomous driving algorithm. During its operation, it continuously reads the latest available unified input representation and calls the autonomous driving algorithm to be evaluated to generate real-time driving decisions.

[0031] S26. The vehicle intelligent agent initiates the underlying vehicle control process, which runs at high frequency as an independent asynchronous process. It continuously reads the latest available real-time driving decisions and generates vehicle control commands based on them. The vehicle control commands include at least one of steering, throttle, and braking.

[0032] S27. The sensor data acquisition process, data preprocessing process, autonomous driving algorithm execution process, and underlying vehicle control process run asynchronously as independent processes, independent threads, running units deployed on independent computing nodes, or a combination thereof, including at least the implementation method of running as independent processes;

[0033] S28. The asynchronous operation mode of the independent process ensures that the simulation world does not stop progressing due to the time consumed by sensor data acquisition, data preprocessing, autonomous driving algorithm execution, or underlying vehicle control calculations, so that the end-to-end operation delay of the autonomous driving algorithm can truly affect the closed-loop driving behavior of the vehicle intelligent agent.

[0034] S29. When the autonomous driving algorithm execution process fails to output a new real-time driving decision within a preset time, the underlying vehicle control process generates a vehicle control command using the most recent valid driving decision, trajectory extrapolation result, safety degradation strategy, smooth braking strategy, or lane keeping strategy.

[0035] To further explain, the execution of the autonomous driving algorithm in step S25 specifically includes:

[0036] S251. Construct a pluggable autonomous driving algorithm access interface, wherein the pluggable autonomous driving algorithm access interface includes an input adaptation unit, an algorithm calling unit, and an output adaptation unit;

[0037] S252, The input adaptation unit reads a unified input representation from shared memory, a cache queue, a publish-subscribe channel, or an inter-process communication channel, and converts it into the input format required by the autonomous driving algorithm to be evaluated;

[0038] S253, The autonomous driving algorithm to be evaluated includes rule control algorithm, modular autonomous driving algorithm, end-to-end neural network model, imitation learning model, reinforcement learning model, online learning model, adaptive model, remote driving algorithm or a combination thereof;

[0039] S254. The algorithm calling unit calls the autonomous driving algorithm to be evaluated based on the converted input format, so that it outputs real-time driving decisions. The real-time driving decisions include at least one of the following: target speed, future trajectory, waypoint sequence, steering angle, throttle command, braking command, lane change command, behavior decision, local path or low-level control command.

[0040] S255. The output adaptation unit receives the real-time driving decisions output by the autonomous driving algorithm to be evaluated, and converts the decision results output by different algorithms into a unified driving decision format. When the unified driving decision format includes the target speed and the future trajectory, the underlying vehicle control process performs longitudinal control based on the target speed and lateral control based on the future trajectory.

[0041] S256. The longitudinal control and lateral control are implemented using proportional-integral-derivative control, pure tracking control, model predictive control, vehicle dynamics control, or a combination thereof.

[0042] S257. Continuously record the input timestamp, output timestamp, inference time, input data version, and output decision version of the autonomous driving algorithm to be evaluated for subsequent execution latency semantic monitoring and performance evaluation.

[0043] To further explain, step S3 specifically involves:

[0044] S31. Start the real-time status acquisition mechanism and establish a frame update callback, status subscription mechanism, event listening mechanism or a combination thereof driven by the simulation server or interactive environment;

[0045] S32. In each state update cycle, parse the current simulation timestamp from the world snapshot object, state message or environment state object provided by the simulation server or interactive environment.

[0046] S33. Obtain real-time status data of the vehicle intelligent agent, wherein the real-time status data includes at least one of real-time pose, velocity vector, acceleration vector, angular velocity, vehicle control status, vehicle bounding box, route progress, and road information where the vehicle is located.

[0047] S34. Obtain real-time status data of background traffic participants, including background vehicles, pedestrians, static obstacles or other traffic participants;

[0048] S35. Obtain traffic signal and road topology information, wherein the traffic signal and road topology information includes at least one of traffic signal light status, stop sign position, lane topology, road boundary, local speed limit, lane centerline and global route information;

[0049] S36. Based on the predefined background traffic participant logical identifiers in the scenario configuration file, match them with the role name attribute or other object attributes set when the simulation vehicle is generated, and obtain the runtime object identifiers dynamically allocated by the simulation server, thereby establishing a mapping dictionary from logical identifiers to runtime object identifiers.

[0050] S37. The mapping dictionary is used to quickly find background traffic participants in subsequent interaction harmony evaluation and continuously track their progress on preset routes, loop routes or task paths.

[0051] S38. The acquired vehicle agent status, background traffic participant status, traffic signal status, road topology information, and simulation timestamp are integrated into a real-time status snapshot and transmitted to the asynchronous multidimensional online evaluation system.

[0052] To further explain, step S3 also includes a pre-calculation process for traffic rule-related objects, specifically:

[0053] S311, Instantiate the traffic light detection module, and scan all or part of the traffic light objects in the map after the simulation world is loaded;

[0054] S312. Pre-calculate static geometric information for each traffic light object, wherein the static geometric information includes the traffic light trigger position, stop line waypoint, stop line vertex coordinates, and traffic light influence area.

[0055] S313. The static geometric information of the traffic lights is cached in a static data structure or index structure for subsequent red light interaction detection and red light violation judgment.

[0056] S314. Instantiate the stop sign detection module and filter candidate stop signs near the vehicle agent from the map object by using a distance threshold.

[0057] S315. Iteratively generate a discrete path point sequence representing the future trajectory of the vehicle agent along the map waypoints. Determine whether the discrete path point sequence intersects with the two-dimensional projection area of ​​the stop sign trigger volume by point using the vector projection method. When the discrete path point sequence intersects with the two-dimensional projection area of ​​the stop sign trigger volume, determine that the corresponding stop sign is a path-related stop sign, and perform vehicle stop determination when the vehicle agent passes the stop sign.

[0058] To further explain, step S4 specifically involves:

[0059] S41. Start the asynchronous multidimensional online evaluation system and create an independent background evaluation process. The background evaluation process is decoupled from the simulation world advancement process, the vehicle intelligent agent asynchronous operation pipeline, and the autonomous driving algorithm execution process.

[0060] S42. The background evaluation process receives or reads the real-time state snapshot generated by the real-time state acquisition mechanism. The real-time state snapshot includes at least one of the following: vehicle agent state, background traffic participant state, traffic signal state, road topology information, vehicle control state, and simulation timestamp.

[0061] S43. The background evaluation process performs online evaluation calculations according to the preset evaluation cycle, simulation frame update cycle, status snapshot arrival event or violation event triggering method, and updates the safety evaluation data, efficiency evaluation data, comfort evaluation data and interaction harmony evaluation data according to the real-time status snapshot.

[0062] S44. The background evaluation process calculates safety evaluation indicators based on safety evaluation data, efficiency evaluation indicators based on route progress and speed limit compliance, comfort evaluation indicators based on vehicle dynamics smoothness and control continuity, and interaction harmony evaluation indicators based on the progress of background traffic participants.

[0063] S45. The background evaluation process integrates the safety evaluation index, efficiency evaluation index, comfort evaluation index and interaction harmony evaluation index according to preset weights to obtain the comprehensive evaluation index at the current moment or the current evaluation window.

[0064] S46. The background evaluation process writes the aforementioned safety evaluation index, efficiency evaluation index, comfort evaluation index, interaction harmony evaluation index, and comprehensive evaluation index into the long-term evaluation record according to the simulation timestamp or physical wall clock timestamp.

[0065] S47. During the online evaluation calculation, the background evaluation process does not output control commands to the vehicle agent, does not participate in the simulation world time progression, and does not block sensor data acquisition, data preprocessing, autonomous driving algorithm execution, and underlying vehicle control processes.

[0066] To further explain, the specific calculation process of the safety evaluation index in step S4 is as follows:

[0067] S411. In the quantitative assessment of safety, the safety evaluation is divided into four sub-indicators: collision detection, emergency braking detection, lane correctness detection, and traffic rule detection.

[0068] S412. In collision detection, a collision sensor is added to the vehicle intelligent agent and a collision event callback function is registered to monitor collision events in real time. A time window filtering method based on collision objects is adopted to maintain a latest collision timestamp for each independent object that collides. When a new collision event is detected, it is only regarded as a new independent collision event if the difference between the timestamp of the event and the latest collision timestamp recorded by the corresponding object exceeds a preset time threshold. The identified independent collision events are classified and stored according to the collision object type, which includes pedestrians, vehicles and static objects.

[0069] S413. In emergency braking detection, the acceleration vector of the vehicle agent in the world coordinate system is obtained and transformed to the vehicle's own local coordinate system to obtain the longitudinal acceleration. When the current speed of the vehicle agent is higher than the preset high speed threshold and the longitudinal acceleration in its local coordinate system is lower than the preset severe deceleration threshold, an emergency braking candidate event is determined to have occurred. The emergency braking candidate event is filtered by a time window method. The current emergency braking event is confirmed and recorded only when the time since the last validly recorded emergency braking event exceeds the preset time window.

[0070] S414. In lane correctness detection, multiple bottom corner points of the vehicle agent bounding box are sampled, and the map waypoints below these sampled points are queried and verified to determine whether the vehicle agent is driving beyond the lane type. The dot product between the vehicle agent's forward vector and the forward vector of its lane or projected waypoint is calculated to determine whether it is driving in reverse. If the dot product is less than a preset threshold, the vehicle agent is determined to be driving in reverse. The shortest Euclidean distance between the vehicle agent's current position and the predetermined global route is calculated. If the distance exceeds a preset route deviation threshold, the route deviation is determined to have occurred.

[0071] S415. In traffic rule detection, a state machine-based delayed penalty method is adopted, delaying the violation determination from the moment the vehicle's behavior occurs until the vehicle has completely left the area affected by the corresponding traffic rule. During interaction with traffic lights, when the front end of the vehicle agent first reaches the stop line, if the traffic light is detected to be red, the state machine is set to a potential violation state. Subsequently, the vehicle agent's position is continuously monitored until the vehicle agent completely leaves the traffic light trigger area. At the moment the vehicle agent leaves, if the traffic light is still red and the state machine was previously in a potential violation state, a red light violation event is confirmed and recorded. During interaction with stop signs, when the vehicle agent enters the stop sign trigger area, the state machine begins to monitor the vehicle speed to determine whether a valid stopping behavior has occurred. When the vehicle agent is detected to have completely left the trigger area, if no valid stopping behavior has occurred during the entire interaction process, a violation event of improper parking is confirmed and recorded.

[0072] S416. Treat collision events and emergency braking events as discrete risk events, and cross-border driving, driving against traffic, and route deviation as persistent violations. Calculate the occurrence rate per kilometer of discrete risk events and the time percentage of persistent violations, and calculate the basic safety score based on a cumulative penalty method. :

[0073]

[0074] in, Discrete risk events The penalty factor For discrete risk events The number of times it occurs, This represents the total mileage traveled by the vehicle. For persistent violations The percentage of time spent on it.

[0075] S417. Calculate the total number of interactions with traffic rules and the number of violations, and calculate traffic rule compliance points. :

[0076]

[0077] in, For traffic rules Total number of interaction opportunities For traffic rules The number of violations.

[0078] S418. The basic safety score and the traffic rule compliance score are weighted and summed to obtain the final safety score. :

[0079]

[0080] in, Preset security fusion weights. This security score serves as a security evaluation indicator.

[0081] To further explain, the specific calculation process of the efficiency evaluation index in step S4 is as follows:

[0082] S421. During the efficiency evaluation process, the current position of the vehicle agent is projected onto the predetermined global route to find the point on the route closest to the vehicle, thereby determining its effective travel distance on the path. This distance is used to characterize the actual degree of progress of the vehicle agent along the predetermined global route;

[0083] S422, Effective forward distance With total path length The ratio is converted to route completion. :

[0084]

[0085] S423. During the evaluation process, the local speed limit of the lane currently occupied by the vehicle agent is continuously obtained through the map interface, road topology interface, or simulation server interface. And obtain the current real-time speed of the vehicle's intelligent agent. Instantaneous velocity score is calculated using a nonlinear penalty function. To punish speeding:

[0086]

[0087] S424. Average the instantaneous velocity scores for all sampling times and convert them to a percentage to obtain the average velocity score:

[0088]

[0089] in, The number of sampling frames during the evaluation process;

[0090] S425. Normalize the route completion score and average speed score, then multiply them to obtain the efficiency score. :

[0091]

[0092] This efficiency score serves as an efficiency evaluation indicator.

[0093] To further explain, the specific calculation process of the comfort evaluation index in step S4 is as follows:

[0094] S431. During the evaluation process, the system maintains a fixed-length first-in-first-out queue for the acceleration vector, throttle control quantity, braking control quantity, and steering control quantity of the vehicle intelligent agent, and performs moving average smoothing on the acceleration vector, throttle control quantity, braking control quantity, and steering control quantity by calculating the arithmetic mean of the data in the first-in-first-out queue.

[0095] S432. Based on the smoothed data, calculate the original indicators for comfort evaluation. These original indicators include the root mean square of longitudinal acceleration, the root mean square of lateral acceleration, the root mean square of the steering rate of change, and the frequency of alternation between throttle and brake commands. The longitudinal acceleration... The following is obtained by approximating the smoothed longitudinal acceleration with respect to time:

[0096]

[0097] S433, For any original comfort index its root mean square Calculated using the following formula:

[0098]

[0099] S434, each original comfort index Input a nonlinear scoring function based on exponential decay to obtain the scores for the corresponding sub-items. :

[0100]

[0101] in, This is the severity coefficient. This is the ideal benchmark value for the corresponding indicator;

[0102] S435. Calculate a weighted average of the scores for multiple comfort sub-items based on preset weights to obtain a comfort score. As a comfort evaluation indicator:

[0103]

[0104] in, For the number of comfort sub-items, For the first The weights of each comfort sub-item are assigned, and the sum of the weights of each comfort sub-item is 1.

[0105] To further explain, the specific calculation process of the interaction harmony evaluation index in step S4 is as follows:

[0106] S441. During the evaluation process, the interaction harmony is calculated by quantifying the impact of the vehicle intelligent agent on the efficiency of background traffic flow.

[0107] S442. The system tracks the progress of each background traffic participant in real time on their preset path, loop path, or task path; for the first... The task completion rate of each background traffic participant is calculated using the waypoint arrival method. :

[0108]

[0109] in, It is the first Background traffic participants at time Number of laps completed Index its current destination waypoint. The total number of waypoints along its path;

[0110] S443. Calculate the arithmetic mean of the task completion rates of all background traffic participants to obtain the average traffic flow completion rate at the current moment. :

[0111]

[0112] in, The number of traffic participants in the background;

[0113] S444. Store the average completion rate of traffic flow along with the current timestamp into the traffic flow time series. At the end of the evaluation, perform a discounted numerical integration on the traffic flow time series to obtain the actual discounted cumulative completion rate. :

[0114]

[0115] in, As a time discount factor, To assess the start time;

[0116] S445. Based on the reference completion rate, the expert system's ideal traffic flow model, or the preset ideal traffic model, calculate the cumulative completion rate of the ideal discount according to the actual timestamp sequence of this assessment. First, scene calibration is performed under conditions free from interference from the tested vehicle's intelligent agent. This ensures that the background traffic participants, test route, traffic signals, road topology, and traffic management policies are consistent with the formal evaluation scenario. The average route completion rate of the background traffic participants over time during the calibration process is statistically analyzed, and the average reference completion rate of the background traffic participants is calculated accordingly. And if this is used as a reference completion rate under ideal conditions, then the ideal reference completion curve is:

[0117]

[0118] The ideal discount cumulative completion rate Calculate using the following formula:

[0119]

[0120] in, To assess the start time, and For adjacent sampling timestamps, This represents the number of sampling points in the traffic flow time series. Time discount factor;

[0121] S446. Calculate the interaction harmony score based on the actual cumulative discount completion rate and the ideal cumulative discount completion rate. As an indicator for evaluating the harmony of interaction:

[0122]

[0123] S447. The interaction harmony score is used to characterize the impact of the vehicle agent on the background traffic flow; when the vehicle agent causes background traffic participants to frequently slow down, stop, detour, or experience a decrease in traffic progress, the interaction harmony score decreases.

[0124] To further explain, step S5 specifically involves:

[0125] S51. During the operation of the vehicle intelligent agent, the end-to-end physical wall clock delay is recorded. The end-to-end physical wall clock delay represents the actual time consumed on the physical hardware between the generation of sensor data and the generation of vehicle control commands.

[0126] S52. During the same end-to-end execution process, record the simulation elapsed delay, whereby the simulation elapsed delay represents the simulation time elapsed within the simulation world during the end-to-end execution process.

[0127] S53. Calculate the delay consistency ratio, which is the ratio between the simulated elapsed delay and the end-to-end physical wall clock delay. The formula is as follows:

[0128]

[0129] in, This represents the latency consistency ratio. To simulate elapsed delay, End-to-end physical wall clock delay;

[0130] S54. Calculate the real-time factor of the simulation system. The real-time factor represents the ratio between the simulation time elapsed in the simulation world and the physical wall clock time in a non-blocking, non-freezing state. It is used to characterize the speed at which the simulation world progresses relative to the physical wall clock time during asynchronous closed-loop interaction. The calculation formula is as follows:

[0131]

[0132] in, For the real-time factors of the simulation system, and These are the simulation times at the beginning and end of the evaluation window, respectively. and These are the physical wall clock times at the start and end of the evaluation window, respectively;

[0133] S55. When the time delay consistency ratio is equal to or close to the real-time factor, it is determined that the actual computational delay of the autonomous driving algorithm to be evaluated is mapped to the closed-loop dynamics.

[0134] S56. When the time delay consistency ratio is close to zero, it is determined that the simulation world has time freeze semantics during the algorithm operation, that is, the algorithm calculation delay has not entered the closed-loop dynamics.

[0135] S57. Calculate the median, long-tailed quantile, maximum value, mean, and variance of the end-to-end physical wall clock delay, and calculate the median, long-tailed quantile, maximum value, mean, and variance of the simulated elapsed delay.

[0136] S58. The latency consistency ratio, real-time factor, simulation frame rate, end-to-end physical wall clock latency, and simulation elapsed latency are used as system-level execution semantic evaluation indicators to evaluate whether the closed-loop simulation system has real latency semantics.

[0137] To further explain, step S6 specifically involves:

[0138] S61. During the long-term closed-loop interaction process, the background evaluation process continuously records the scores of each dimension according to the timestamp. The scores of each dimension include security score, efficiency score, comfort score, and interaction harmony score. A comprehensive score is calculated based on the scores of each dimension, and the comprehensive score is calculated according to the following formula:

[0139]

[0140] in, , , and The weights are assigned to safety, efficiency, comfort, and interaction harmony, respectively, and the sum of each weight is 1.

[0141] S62. The background evaluation process continuously records specific violations and the frequency of violations per unit distance. The specific violations include at least one of pedestrian collisions, vehicle collisions, static object collisions, emergency braking, leaving the effective lane, driving in the wrong direction, deviating from the route, running a red light, and improper parking. The frequency of violations per unit distance includes the frequency of occurrence of various collision events and emergency braking events per kilometer.

[0142] S63. The background evaluation process continuously records the background traffic flow time series, which includes the task completion rate of each background traffic participant, the average completion rate of the traffic flow, the timestamp, and the intermediate amount of the discount points.

[0143] S64. The background evaluation process continuously records latency semantic data, which includes end-to-end physical wall clock latency, simulated elapsed latency, latency consistency ratio, simulated frame rate, and real-time factor.

[0144] S65. The background evaluation process updates the above-mentioned recorded data according to fixed time intervals, fixed frame intervals or event triggering methods, and performs statistical summarization on the comprehensive score, scores of each dimension, violation frequency per unit distance, background traffic flow time series and time delay semantic data to generate real-time evaluation curves, phased evaluation results or statistical results for report generation.

[0145] S66. Determine whether the long-range closed-loop interaction process meets the termination conditions. When at least one of the following termination conditions is met: fixed physical wall clock time window, fixed simulation time window, fixed route task, or fixed number of interaction events, terminate the long-range closed-loop interaction process.

[0146] S67. After the evaluation process is completed, the system generates a comprehensive performance evaluation report based on the recorded data and statistical summary results. The comprehensive performance evaluation report includes a comprehensive score, scores for each dimension, violation index per unit distance, raw calculation index, latency semantic index, and system throughput index. Among them, the raw calculation index includes route completion degree, average speed score, root mean square value of various comfort indexes, number of opportunities to comply with traffic rules, number of violations, rule compliance success rate, cumulative completion degree of background traffic flow, and normalized intermediate quantity of interaction harmony.

[0147] S68. The system adopts a separate storage strategy, saving the comprehensive performance evaluation report as the main report file and saving the background traffic flow time series, delay time series, frame-by-frame evaluation index or other large-scale raw time series data as independent data files. The main report file and independent data files are stored using JSON, CSV, Parquet, database tables or a combination thereof.

[0148] S69. The system generates a visual evaluation curve, an algorithm comparison table, or a phased performance summary based on the comprehensive performance evaluation report, which is used to support the horizontal comparison of different autonomous driving algorithms in the same asynchronous closed-loop simulation environment.

[0149] The beneficial effects of this invention are:

[0150] 1. This invention decouples the simulation propulsion load from the intelligent agent's computational load through an asynchronous multi-process architecture, breaking through the time freeze bottleneck of synchronous simulation, constructing continuous time semantics, so that the end-to-end physical computation latency of autonomous driving algorithms can be realistically mapped to closed-loop dynamics, improving the reference value of simulation testing in the real physical world.

[0151] 2. This invention designs a background evaluation system independent of the main process and establishes a multi-dimensional online evaluation system. In addition to basic safety and efficiency indicators, it innovatively introduces a comfort index based on vehicle dynamics and an interaction harmony index based on discounted time integral, which can macroscopically quantify passenger comfort and the impact of autonomous vehicles on overall traffic flow.

[0152] 3. This invention introduces an execution latency semantic monitoring mechanism, which can dynamically monitor the latency fidelity of the system by calculating the latency consistency ratio and real-time factor in real time. At the same time, it incorporates security degradation and fallback strategies to ensure the system stability during long-range closed-loop interaction and online continuous learning.

[0153] 4. This invention can be applied to closed-loop simulation testing of autonomous driving algorithms, end-to-end autonomous driving model verification, online learning algorithm evaluation, long-range driving behavior analysis, stress testing of complex traffic scenarios, and performance comparison of different autonomous driving algorithms in a unified asynchronous closed-loop simulation environment. Attached Figure Description

[0154] Figure 1 A schematic diagram of the overall process of this invention;

[0155] Figure 2 A schematic diagram of the interaction architecture between the asynchronous operation pipeline and the background evaluation process of the vehicle intelligent agent of the present invention;

[0156] Figure 3 A schematic diagram of the multidimensional online evaluation process of this invention;

[0157] Figure 4 A schematic diagram of the latency semantic monitoring process of this invention; Detailed Implementation

[0158] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0159] like Figure 1 As shown, this invention is a method and system for closed-loop simulation and multi-dimensional online evaluation of autonomous driving based on an asynchronous architecture. In this embodiment, the asynchronous closed-loop interaction environment for autonomous driving is built based on the CARLA simulator. The asynchronous closed-loop interaction logic, unified signal system, communication interface, sensor callback, state subscription, vehicle control, online evaluation process, etc., are implemented based on Python, including the following implementation steps:

[0160] S1. Establish an asynchronous closed-loop interactive environment for autonomous driving, initialize the simulation world, vehicle intelligent agent, test route, background traffic participants and system communication interface, and set the simulation world to advance with continuous time semantics.

[0161] S2. Configure an asynchronous operation pipeline for the vehicle intelligent agent, so that sensor data acquisition, data preprocessing, autonomous driving algorithm execution and underlying vehicle control can work together in an asynchronous multi-process manner without blocking the progress of the simulation world.

[0162] S3. Run an event-driven real-time state acquisition mechanism, establish a state subscription mechanism driven by a simulation server or interactive environment, and asynchronously acquire real-time state snapshots of vehicle intelligent agents, background traffic participants, traffic signals and road topology.

[0163] S4. Start the asynchronous multidimensional online evaluation system, run an independent background evaluation process, and calculate the evaluation indicators of safety, efficiency, comfort and interaction harmony based on real-time status snapshots.

[0164] S5. Introduce an execution latency semantic monitoring mechanism to record end-to-end physical wall clock latency and simulation elapsed latency, and calculate the latency consistency ratio to determine whether the computation latency of the autonomous driving algorithm to be evaluated is mapped to the closed-loop dynamics.

[0165] S6. During the long-range closed-loop interaction process, continuously record multi-dimensional evaluation data, violation event data, background traffic flow data, and time delay semantic data, and generate a multi-dimensional autonomous driving performance evaluation report after the evaluation is completed.

[0166] To further explain, step S1 specifically includes:

[0167] Establish an asynchronous closed-loop interactive environment for autonomous driving, initialize the simulation world, vehicle intelligent agent, test route, background traffic participants and system communication interface, and set the simulation world to advance with continuous time semantics;

[0168] S11. Start the autonomous driving closed-loop simulation system, load the autonomous driving simulation world, and initialize the test map, test route, traffic signals, road topology, background traffic participants, the main control vehicle corresponding to the vehicle intelligent agent, and system operating parameters.

[0169] S12. Create at least one vehicle intelligent agent and configure the vehicle intelligent agent with a sensor kit, vehicle status interface, navigation interface, control interface, autonomous driving algorithm interface and evaluation data interface;

[0170] S13. Establish a communication connection with the simulation server or interactive environment, and obtain interface objects for simulation world state query, map information query, vehicle generation, traffic participant management, sensor management and event listening.

[0171] S14. Set the simulation environment stepping mode to asynchronous mode and configure the minimum stepping interval so that the simulation world advances with continuous time semantics, and its time advancement does not need to wait for the vehicle intelligent agent to complete sensor data acquisition, data preprocessing, autonomous driving algorithm execution or underlying vehicle control.

[0172] S15. The simulation world is deployed on the first computing node, and the asynchronous operation pipeline of the vehicle intelligence agent is deployed on the second computing node (here, the vehicle intelligence agent refers to one or more vehicle intelligence agents created in step S12. This simulation system supports parallel testing of multiple vehicle intelligence agents. All vehicle intelligence agents are equipped with the autonomous driving algorithm to be evaluated and can be configured with an asynchronous operation pipeline; if the simulation object is not equipped with the autonomous driving algorithm to be evaluated, but only runs according to a preset route, loop route, behavior strategy, or traffic management strategy, it is classified as a background traffic participant). The first computing node and the second computing node interact with each other through Ethernet, LAN, or wireless network communication links, so that the simulation world advancement task and the vehicle intelligence agent computing task are executed in parallel on different computing nodes, decoupling the simulation load and the computing load, and reducing the impact of the vehicle intelligence agent's computing time on the continuous time advancement of the simulation world.

[0173] S16. Initialize the unified signal system, which is used to broadcast global control signals to the sensor data acquisition process, data preprocessing process, autonomous driving algorithm execution process, underlying vehicle control process and background evaluation process when the scene is initialized, the scene is reset, the simulation starts, the simulation is paused, the abnormal interruption occurs, the evaluation ends and the system is terminated.

[0174] S17. Configure preset routes, loop routes, behavioral strategies or traffic management strategies for background traffic participants, and set discrete route checkpoints for background traffic participants so that background traffic participants can maintain local interaction degrees of freedom while maintaining the consistency of macro traffic flow structure in repeated tests.

[0175] S18. Initialize the data structure for storing evaluation data. The data structure includes a master vehicle indicator data structure for storing the state and evaluation indicators of the vehicle intelligent agent, a global traffic indicator data structure for storing the state of background traffic participants and traffic flow time series, and a violation log data structure for recording specific violation events.

[0176] To further explain, step S2 specifically involves:

[0177] Configure an asynchronous pipeline for the vehicle's intelligent agent, enabling sensor data acquisition, data preprocessing, autonomous driving algorithm execution, and underlying vehicle control to operate collaboratively in an asynchronous, multi-process manner without hindering the progress of the simulation world. Figure 2 As shown;

[0178] S21. Start the sensor data acquisition process for the vehicle intelligent agent. The sensor data acquisition process asynchronously acquires multimodal sensor data according to the sampling frequency of each sensor. The data includes at least one of the following: front view image, multi-view image, lidar point cloud, millimeter-wave radar data, inertial measurement data, global navigation satellite system data, vehicle speed, vehicle attitude, chassis status, navigation commands, and historical trajectory.

[0179] S22. The sensor data acquisition process continuously writes sensor data through callback mechanism, publish-subscribe mechanism, shared cache, shared memory or message queue. The data generation process of different sensors does not need to be completed synchronously in the same simulation frame or the same physical moment.

[0180] S23. Initiate a data preprocessing process for the vehicle intelligent agent. This data preprocessing process, acting as an asynchronous consumer of sensor data, continuously reads the latest available multimodal sensor data and performs timestamp calibration, multi-source data alignment, coordinate system transformation, normalization, image preprocessing, point cloud preprocessing, vehicle state encoding, and navigation command encoding on the data to generate a unified input representation. This unified input representation includes image tensors, point cloud tensors, bird's-eye view features, vehicle state vectors, navigation commands, historical motion states, or combinations thereof, used to mask differences between different sensor input formats.

[0181] S24. The data preprocessing process writes the unified input representation into shared memory, cache queue, publish-subscribe channel or inter-process communication channel for the autonomous driving algorithm execution process to read.

[0182] S25. The vehicle intelligent agent initiates the execution process of the autonomous driving algorithm. During its operation, it continuously reads the latest available unified input representation and calls the autonomous driving algorithm to be evaluated to generate real-time driving decisions.

[0183] S26. The vehicle intelligent agent initiates the underlying vehicle control process, which runs at high frequency as an independent asynchronous process. It continuously reads the latest available real-time driving decisions and generates vehicle control commands based on them. The vehicle control commands include at least one of steering, throttle, and braking.

[0184] S27. The sensor data acquisition process, data preprocessing process, autonomous driving algorithm execution process, and underlying vehicle control process run asynchronously as independent processes, independent threads, running units deployed on independent computing nodes, or a combination thereof, including at least the implementation method of running as independent processes;

[0185] S28. The asynchronous operation mode of the independent process ensures that the simulation world does not stop progressing due to the time consumed by sensor data acquisition, data preprocessing, autonomous driving algorithm execution, or underlying vehicle control calculations, so that the end-to-end operation delay of the autonomous driving algorithm can truly affect the closed-loop driving behavior of the vehicle intelligent agent.

[0186] S29. When the autonomous driving algorithm execution process fails to output a new real-time driving decision within a preset time, the underlying vehicle control process generates a vehicle control command using the most recent valid driving decision, trajectory extrapolation result, safety degradation strategy, smooth braking strategy, or lane keeping strategy.

[0187] To further explain, the execution of the autonomous driving algorithm in step S2 specifically includes:

[0188] S211. Construct a pluggable autonomous driving algorithm access interface, wherein the pluggable autonomous driving algorithm access interface includes an input adaptation unit, an algorithm calling unit, and an output adaptation unit;

[0189] S212, The input adaptation unit reads a unified input representation from shared memory, a cache queue, a publish-subscribe channel, or an inter-process communication channel, and converts it into the input format required by the autonomous driving algorithm to be evaluated;

[0190] S213, The autonomous driving algorithm to be evaluated includes rule control algorithm, modular autonomous driving algorithm, end-to-end neural network model, imitation learning model, reinforcement learning model, online learning model, adaptive model, remote driving algorithm or a combination thereof;

[0191] S214. The algorithm calling unit calls the autonomous driving algorithm to be evaluated based on the converted input format, so that it outputs real-time driving decisions. The real-time driving decisions include at least one of the following: target speed, future trajectory, waypoint sequence, steering angle, throttle command, braking command, lane change command, behavior decision, local path or low-level control command.

[0192] S215. The output adaptation unit receives the real-time driving decisions output by the autonomous driving algorithm to be evaluated, and converts the decision results output by different algorithms into a unified driving decision format. When the unified driving decision format includes the target speed and the future trajectory, the underlying vehicle control process performs longitudinal control based on the target speed and lateral control based on the future trajectory.

[0193] S216. The longitudinal control and lateral control are implemented using proportional-integral-derivative control, pure tracking control, model predictive control, vehicle dynamics control, or a combination thereof.

[0194] S217. Continuously record the input timestamp, output timestamp, inference time, input data version, and output decision version of the autonomous driving algorithm to be evaluated for subsequent execution latency semantic monitoring and performance evaluation.

[0195] To further explain, step S3 specifically involves:

[0196] Run an event-driven real-time state acquisition mechanism and establish a state subscription mechanism driven by a simulation server or interactive environment to asynchronously acquire real-time state snapshots of vehicle agents, background traffic participants, traffic signals, and road topology.

[0197] S31. Start the real-time status acquisition mechanism and establish a frame update callback, status subscription mechanism, event listening mechanism or a combination thereof driven by the simulation server or interactive environment;

[0198] S32. In each state update cycle, parse the current simulation timestamp from the world snapshot object, state message or environment state object provided by the simulation server or interactive environment.

[0199] S33. Obtain real-time status data of the vehicle intelligent agent, wherein the real-time status data includes at least one of real-time pose, velocity vector, acceleration vector, angular velocity, vehicle control status, vehicle bounding box, route progress, and road information where the vehicle is located.

[0200] S34. Obtain real-time status data of background traffic participants, including background vehicles, pedestrians, static obstacles or other traffic participants;

[0201] S35. Obtain traffic signal and road topology information, wherein the traffic signal and road topology information includes at least one of traffic signal light status, stop sign position, lane topology, road boundary, local speed limit, lane centerline and global route information;

[0202] S36. Based on the predefined background traffic participant logical identifiers in the scenario configuration file, match them with the role name attribute or other object attributes set when the simulation vehicle is generated, and obtain the runtime object identifiers dynamically allocated by the simulation server, thereby establishing a mapping dictionary from logical identifiers to runtime object identifiers.

[0203] S37. The mapping dictionary is used to quickly find background traffic participants in subsequent interaction harmony evaluation and continuously track their progress on preset routes, loop routes or task paths.

[0204] S38. Integrate the acquired vehicle agent status, background traffic participant status, traffic signal status, road topology information, and simulation timestamp into a real-time status snapshot and transmit it to the asynchronous multidimensional online evaluation system.

[0205] To further explain, step S3 also includes a pre-calculation process for traffic rule-related objects, specifically:

[0206] S311, Instantiate the traffic light detection module, and scan all or part of the traffic light objects in the map after the simulation world is loaded;

[0207] S312. Pre-calculate static geometric information for each traffic light object, wherein the static geometric information includes the traffic light trigger position, stop line waypoint, stop line vertex coordinates, and traffic light influence area.

[0208] S313. The static geometric information of the traffic lights is cached in a static data structure or index structure for subsequent red light interaction detection and red light violation judgment.

[0209] S314. Instantiate the stop sign detection module and filter candidate stop signs near the vehicle agent from the map object by using a distance threshold.

[0210] S315. Iteratively generate a discrete path point sequence representing the future trajectory of the vehicle agent along the map waypoints. Determine whether the discrete path point sequence intersects with the two-dimensional projection area of ​​the stop sign trigger volume by point using the vector projection method. When the discrete path point sequence intersects with the two-dimensional projection area of ​​the stop sign trigger volume, determine that the corresponding stop sign is a path-related stop sign, and perform vehicle stop determination when the vehicle agent passes the stop sign.

[0211] To further explain, step S4 specifically involves:

[0212] Launch an asynchronous multidimensional online evaluation system, run an independent background evaluation process, and calculate evaluation indicators for safety, efficiency, comfort, and interaction harmony based on real-time status snapshots, such as... Figure 3 As shown;

[0213] S41. Start the asynchronous multidimensional online evaluation system and create an independent background evaluation process. The background evaluation process is decoupled from the simulation world advancement process, the vehicle intelligent agent asynchronous operation pipeline, and the autonomous driving algorithm execution process.

[0214] S42. The background evaluation process receives or reads the real-time state snapshot generated by the real-time state acquisition mechanism. The real-time state snapshot includes at least one of the following: vehicle agent state, background traffic participant state, traffic signal state, road topology information, vehicle control state, and simulation timestamp.

[0215] S43. The background evaluation process performs online evaluation calculations according to the preset evaluation cycle, simulation frame update cycle, status snapshot arrival event or violation event triggering method, and updates the safety evaluation data, efficiency evaluation data, comfort evaluation data and interaction harmony evaluation data according to the real-time status snapshot.

[0216] S44. The background evaluation process calculates safety evaluation indicators based on safety evaluation data, efficiency evaluation indicators based on route progress and speed limit compliance, comfort evaluation indicators based on vehicle dynamics smoothness and control continuity, and interaction harmony evaluation indicators based on the progress of background traffic participants.

[0217] S45. The background evaluation process integrates the security evaluation index, efficiency evaluation index, comfort evaluation index and interaction harmony evaluation index according to preset weights to obtain the comprehensive evaluation index at the current moment or the current evaluation window.

[0218] S46. The background evaluation process writes the safety evaluation index, efficiency evaluation index, comfort evaluation index, interaction harmony evaluation index and comprehensive evaluation index into the long-term evaluation record according to the simulation timestamp or physical wall clock timestamp.

[0219] S47. During the online evaluation calculation, the background evaluation process does not output control commands to the vehicle intelligent agent, does not participate in the simulation world time progression, and does not block sensor data acquisition, data preprocessing, autonomous driving algorithm execution, and underlying vehicle control processes.

[0220] To further explain, the specific calculation process of the safety evaluation index in step S4 is as follows:

[0221] S411. In the quantitative assessment of safety, the safety evaluation is divided into four sub-indicators: collision detection, emergency braking detection, lane correctness detection, and traffic rule detection.

[0222] S412. In collision detection, a collision sensor is added to the vehicle intelligent agent and a collision event callback function is registered to monitor collision events in real time. A time window filtering method based on collision objects is adopted to maintain a latest collision timestamp for each independent object that collides. When a new collision event is detected, it is only regarded as a new independent collision event if the difference between the timestamp of the event and the latest collision timestamp recorded by the corresponding object exceeds a preset time threshold. The identified independent collision events are classified and stored according to the collision object type, which includes pedestrians, vehicles and static objects.

[0223] S413. In emergency braking detection, the acceleration vector of the vehicle agent in the world coordinate system is obtained and transformed to the vehicle's own local coordinate system to obtain the longitudinal acceleration. When the current speed of the vehicle agent is higher than the preset high speed threshold and the longitudinal acceleration in its local coordinate system is lower than the preset severe deceleration threshold, an emergency braking candidate event is determined to have occurred. The emergency braking candidate event is filtered by a time window method. The current emergency braking event is confirmed and recorded only when the time since the last validly recorded emergency braking event exceeds the preset time window.

[0224] S414. In lane correctness detection, multiple bottom corner points of the vehicle agent bounding box are sampled, and the map waypoints below these sampled points are queried and verified to determine whether the vehicle agent is driving beyond the lane type. The dot product between the vehicle agent's forward vector and the forward vector of its lane or projected waypoint is calculated to determine whether it is driving in reverse. If the dot product is less than a preset threshold, the vehicle agent is determined to be driving in reverse. The shortest Euclidean distance between the vehicle agent's current position and the predetermined global route is calculated. If the distance exceeds a preset route deviation threshold, the route deviation is determined to have occurred.

[0225] S415. In traffic rule detection, a state machine-based delayed penalty method is adopted, delaying the violation determination from the moment the vehicle's behavior occurs until the vehicle has completely left the area affected by the corresponding traffic rule. During interaction with traffic lights, when the front end of the vehicle agent first reaches the stop line, if the traffic light is detected to be red, the state machine is set to a potential violation state. Subsequently, the vehicle agent's position is continuously monitored until the vehicle agent completely leaves the traffic light trigger area. At the moment the vehicle agent leaves, if the traffic light is still red and the state machine was previously in a potential violation state, a red light violation event is confirmed and recorded. During interaction with stop signs, when the vehicle agent enters the stop sign trigger area, the state machine begins to monitor the vehicle speed to determine whether a valid stopping behavior has occurred. When the vehicle agent is detected to have completely left the trigger area, if no valid stopping behavior has occurred during the entire interaction process, a violation event of improper parking is confirmed and recorded.

[0226] S416. Treat collision events and emergency braking events as discrete risk events, and cross-border driving, driving against traffic, and route deviation as continuous violations. Calculate the occurrence rate per kilometer of discrete risk events and the time percentage of continuous violations, and calculate a basic safety score based on a cumulative penalty method:

[0227]

[0228] in, For discrete risk events The penalty factor For discrete risk events The number of times it occurs, This represents the total mileage traveled by the vehicle. For persistent violations The percentage of time spent on it.

[0229] S417. Calculate the total number of interactions with traffic rules and the number of violations, and then calculate the traffic rule compliance score:

[0230]

[0231] in, For traffic rules Total number of interaction opportunities For traffic rules The number of violations.

[0232] S418. The basic safety score and the traffic rule compliance score are weighted and summed to obtain the final safety score:

[0233]

[0234] in, Preset security fusion weights;

[0235] To further explain, the specific calculation process of the efficiency evaluation index in step S4 is as follows:

[0236] S421. During the efficiency evaluation process, the current position of the vehicle agent is projected onto the predetermined global route to find the point on the route closest to the vehicle, thereby determining its effective travel distance on the path. This distance is used to characterize the actual degree of progress of the vehicle agent along the predetermined global route;

[0237] S422, Effective forward distance With total path length The ratio is converted to route completion rate:

[0238]

[0239] S423. During the evaluation process, the local speed limit of the lane currently occupied by the vehicle agent is continuously obtained through the map interface, road topology interface, or simulation server interface. And obtain the current real-time speed of the vehicle's intelligent agent. Instantaneous speed score is calculated using a nonlinear penalty function to penalize speeding behavior:

[0240]

[0241] S424. Average the instantaneous velocity scores for all sampling times and convert them to a percentage to obtain the average velocity score:

[0242]

[0243] in, The number of sampling frames during the evaluation process;

[0244] S425. Normalize the route completion score and multiply it by the average speed score to obtain the efficiency score:

[0245]

[0246] To further explain, the specific calculation process of the comfort evaluation index in step S4 is as follows:

[0247] S431. During the evaluation process, the system maintains a fixed-length first-in-first-out queue for the acceleration vector, throttle control quantity, braking control quantity, and steering control quantity of the vehicle intelligent agent, and performs moving average smoothing on the acceleration vector, throttle control quantity, braking control quantity, and steering control quantity by calculating the arithmetic mean of the data in the first-in-first-out queue.

[0248] S432. Based on the smoothed data, calculate the original indicators for comfort evaluation. These original indicators include the root mean square of longitudinal acceleration, the root mean square of lateral acceleration, the root mean square of the steering rate of change, and the frequency of alternation between throttle and brake commands. The longitudinal acceleration is approximated by the difference between the smoothed longitudinal acceleration and time.

[0249]

[0250] S433, For any original comfort index Its root mean square is calculated using the following formula:

[0251]

[0252] S434, each original comfort index Input a non-linear scoring function based on exponential decay to obtain the scores for the corresponding sub-items:

[0253]

[0254] in, This is the severity coefficient. This is the ideal benchmark value for the corresponding indicator;

[0255] S435. Calculate the weighted average of the scores for multiple comfort sub-items based on preset weights to obtain the comfort score:

[0256]

[0257] in, For the number of comfort sub-items, For the first The weights of each comfort sub-item are assigned, and the sum of the weights of each comfort sub-item is 1.

[0258] To further explain, the specific calculation process of the interaction harmony evaluation index in step S4 is as follows:

[0259] S441. During the evaluation process, the interaction harmony is calculated by quantifying the impact of the vehicle intelligent agent on the efficiency of background traffic flow.

[0260] S442. The system tracks the progress of each background traffic participant in real time on their preset path, loop path, or task path; for the first... For each background traffic participant, their task completion rate is calculated using the waypoint arrival method:

[0261]

[0262] in, It is the first Background traffic participants at time Number of laps completed Index its current destination waypoint. The total number of waypoints along its path;

[0263] S443. Calculate the arithmetic mean of the task completion rates of all background traffic participants to obtain the average traffic flow completion rate at the current moment:

[0264]

[0265] in, The number of traffic participants in the background;

[0266] S444. Store the average completion rate of traffic flow along with the current timestamp into the traffic flow time series. At the end of the evaluation, perform a discounted numerical integration on the traffic flow time series to obtain the actual discounted cumulative completion rate:

[0267]

[0268] in, As a time discount factor, To assess the start time;

[0269] S445. Based on the reference completion rate, the expert system's ideal traffic flow model, or the preset ideal traffic model, calculate the cumulative completion rate of the ideal discount according to the actual timestamp sequence of this assessment. ;

[0270] S446. Calculate the interaction harmony score based on the actual cumulative discount completion rate and the ideal cumulative discount completion rate:

[0271]

[0272] S447. The interaction harmony score is used to characterize the impact of the vehicle agent on the background traffic flow; when the vehicle agent causes background traffic participants to frequently slow down, stop, detour, or experience a decrease in traffic progress, the interaction harmony score decreases.

[0273] To further explain, step S5 specifically involves:

[0274] An execution latency semantic monitoring mechanism is introduced to record the end-to-end physical wall clock latency and the simulation elapsed latency, and to calculate the latency consistency ratio, in order to determine whether the computation latency of the autonomous driving algorithm under evaluation is mapped to the closed-loop dynamics, such as... Figure 4 As shown;

[0275] S51. During the operation of the vehicle intelligent agent, the end-to-end physical wall clock delay is recorded. The end-to-end physical wall clock delay represents the actual time consumed on the physical hardware between the generation of sensor data and the generation of vehicle control commands.

[0276] S52. During the same end-to-end execution process, record the simulation elapsed delay, whereby the simulation elapsed delay represents the simulation time elapsed within the simulation world during the end-to-end execution process.

[0277] S53. Calculate the delay consistency ratio, which is the ratio between the simulated elapsed delay and the end-to-end physical wall clock delay. The formula is as follows:

[0278]

[0279] in, This represents the latency consistency ratio. To simulate elapsed delay, End-to-end physical wall clock delay;

[0280] S54. Calculate the real-time factor of the simulation system, wherein the real-time factor represents the ratio between the simulation time elapsed in the simulated world under unblocked and unfrozen conditions and the physical wall clock time.

[0281] S55. When the time delay consistency ratio is equal to or close to the real-time factor, it is determined that the actual computational delay of the autonomous driving algorithm to be evaluated is mapped to the closed-loop dynamics.

[0282] S56. When the time delay consistency ratio is close to zero, it is determined that the simulation world has time freeze semantics during the algorithm operation, that is, the algorithm calculation delay has not entered the closed-loop dynamics.

[0283] S57. Calculate the median, long-tailed quantile, maximum value, mean, and variance of the end-to-end physical wall clock delay, and calculate the median, long-tailed quantile, maximum value, mean, and variance of the simulated elapsed delay.

[0284] S58. The latency consistency ratio, real-time factor, simulation frame rate, end-to-end physical wall clock latency, and simulation elapsed latency are used as system-level execution semantic evaluation indicators to evaluate whether the closed-loop simulation system has real latency semantics.

[0285] To further explain, step S6 specifically involves:

[0286] During the long-range closed-loop interaction process, multi-dimensional evaluation data, violation event data, background traffic flow data and time delay semantic data are continuously recorded, and a multi-dimensional autonomous driving performance evaluation report is generated after the evaluation is completed.

[0287] S61. During the long-term closed-loop interaction process, the background evaluation process continuously records the scores of each dimension according to the timestamp. The scores of each dimension include security score, efficiency score, comfort score, and interaction harmony score. A comprehensive score is calculated based on the scores of each dimension, and the comprehensive score is calculated according to the following formula:

[0288]

[0289] in, , , and The weights are assigned to safety, efficiency, comfort, and interaction harmony, respectively, and the sum of each weight is 1.

[0290] S62. The background evaluation process continuously records specific violations and the frequency of violations per unit distance. The specific violations include at least one of pedestrian collisions, vehicle collisions, static object collisions, emergency braking, leaving the effective lane, driving in the wrong direction, deviating from the route, running a red light, and improper parking. The frequency of violations per unit distance includes the frequency of occurrence of various collision events and emergency braking events per kilometer.

[0291] S63. The background evaluation process continuously records the background traffic flow time series, which includes the task completion rate of each background traffic participant, the average completion rate of the traffic flow, the timestamp, and the intermediate amount of the discount points.

[0292] S64. The background evaluation process continuously records latency semantic data, which includes end-to-end physical wall clock latency, simulated elapsed latency, latency consistency ratio, simulated frame rate, and real-time factor.

[0293] S65. The background evaluation process updates the above-mentioned recorded data according to fixed time intervals, fixed frame intervals or event triggering methods, and performs statistical summarization on the comprehensive score, scores of each dimension, violation frequency per unit distance, background traffic flow time series and time delay semantic data to generate real-time evaluation curves, phased evaluation results or statistical results for report generation.

[0294] S66. Determine whether the long-range closed-loop interaction process meets the termination conditions. When at least one of the following termination conditions is met: fixed physical wall clock time window, fixed simulation time window, fixed route task, or fixed number of interaction events, terminate the long-range closed-loop interaction process.

[0295] S67. After the evaluation process is completed, the system generates a comprehensive performance evaluation report based on the recorded data and statistical summary results. The comprehensive performance evaluation report includes a comprehensive score, scores for each dimension, violation index per unit distance, raw calculation index, latency semantic index, and system throughput index. Among them, the raw calculation index includes route completion degree, average speed score, root mean square value of various comfort indexes, number of opportunities to comply with traffic rules, number of violations, rule compliance success rate, cumulative completion degree of background traffic flow, and normalized intermediate quantity of interaction harmony.

[0296] S68. The system adopts a separate storage strategy, saving the comprehensive performance evaluation report as the main report file and saving the background traffic flow time series, delay time series, frame-by-frame evaluation index or other large-scale raw time series data as independent data files. The main report file and independent data files are stored using JSON, CSV, Parquet, database tables or a combination thereof.

[0297] S69. The system generates a visual evaluation curve, an algorithm comparison table, or a phased performance summary based on the comprehensive performance evaluation report, which is used to support the horizontal comparison of different autonomous driving algorithms in the same asynchronous closed-loop simulation environment.

[0298] It should be noted that this specification describes the implementation methods, but not every specific step has an independent technical solution. This explanatory mode is used to help readers better understand the design steps and various methods used in this invention. Those skilled in the art should consider the entire specification as a whole; various technical solutions can be appropriately combined to form other implementation methods that can be understood by those skilled in the art.

[0299] The detailed descriptions listed above are merely specific descriptions of feasible embodiments of the present invention, and are not intended to limit the scope of protection of the present invention. All equivalent methods or modifications that do not depart from the technology of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for closed-loop simulation and multi-dimensional online evaluation of autonomous driving based on asynchronous architecture, characterized in that, include: S1. Establish an asynchronous closed-loop interactive environment for autonomous driving, initialize the simulation world, vehicle intelligent agent, test route, background traffic participants and communication interface, and set the simulation world to advance with continuous time semantics. S2. Configure an asynchronous operation pipeline for the vehicle intelligent agent, so that sensor data acquisition, data preprocessing, autonomous driving algorithm execution and underlying vehicle control can work together in an asynchronous multi-process manner without blocking the progress of the simulation world. S3. Run an event-driven real-time state acquisition mechanism, establish a state subscription mechanism driven by a simulation server or interactive environment, and asynchronously acquire real-time state snapshots of vehicle intelligent agents, background traffic participants, traffic signals and road topology. S4. Start the asynchronous multidimensional online evaluation system, run an independent background evaluation process, and calculate the evaluation indicators of safety, efficiency, comfort and interaction harmony based on real-time status snapshots. S5. Introduce an execution latency semantic monitoring mechanism to record end-to-end physical wall clock latency and simulation elapsed latency, and calculate the latency consistency ratio to determine whether the computation latency of the autonomous driving algorithm to be evaluated is mapped to the closed-loop dynamics. S6. During the long-range closed-loop interaction process, continuously record multi-dimensional evaluation data, violation event data, background traffic flow data, and time delay semantic data, and generate a multi-dimensional autonomous driving performance evaluation report after the evaluation is completed.

2. The asynchronous architecture-based closed-loop simulation and multi-dimensional online evaluation method for autonomous driving according to claim 1, characterized in that, Step S1 specifically involves: S11. Start the autonomous driving closed-loop simulation system, load the autonomous driving simulation world, and initialize the test map, test route, traffic signals, road topology, background traffic participants, the main control vehicle corresponding to the vehicle intelligent agent, and system operating parameters. S12. Create at least one vehicle intelligent agent and configure the vehicle intelligent agent with a sensor kit, vehicle status interface, navigation interface, control interface, autonomous driving algorithm interface and evaluation data interface; S13. Establish a communication connection with the simulation server or interactive environment, and obtain interface objects for simulation world state query, map information query, vehicle generation, traffic participant management, sensor management and event listening. S14. Set the simulation environment stepping mode to asynchronous mode and configure the minimum stepping interval so that the simulation world advances with continuous time semantics, and its time advancement does not need to wait for the vehicle intelligent agent to complete sensor data acquisition, data preprocessing, autonomous driving algorithm execution or underlying vehicle control. S15. The simulation world is deployed on the first computing node, and the asynchronous operation pipeline of the vehicle intelligent agent is deployed on the second computing node. The first computing node and the second computing node interact with each other through Ethernet, LAN or wireless network communication links, so that the simulation world advancement task and the vehicle intelligent agent computing task are executed in parallel on different computing nodes, decoupling the simulation load and the computing load, and reducing the impact of the vehicle intelligent agent's computing time on the continuous time advancement of the simulation world. S16. Initialize the unified signal system, which is used to broadcast global control signals to the sensor data acquisition process, data preprocessing process, autonomous driving algorithm execution process, underlying vehicle control process and background evaluation process when the scene is initialized, the scene is reset, the simulation starts, the simulation is paused, the abnormal interruption occurs, the evaluation ends and the system is terminated. S17. Configure preset routes, loop routes, behavioral strategies or traffic management strategies for background traffic participants, and set discrete route checkpoints for background traffic participants so that background traffic participants can maintain local interaction degrees of freedom while maintaining the consistency of macro traffic flow structure in repeated tests. S18. Initialize the data structure for storing evaluation data. The data structure includes a master vehicle indicator data structure for storing the state and evaluation indicators of the vehicle intelligent agent, a global traffic indicator data structure for storing the state of background traffic participants and traffic flow time series, and a violation log data structure for recording specific violation events.

3. The asynchronous architecture-based closed-loop simulation and multi-dimensional online evaluation method for autonomous driving according to claim 2, characterized in that, Step S2 specifically involves: S21. Start the sensor data acquisition process for the vehicle intelligent agent. The sensor data acquisition process asynchronously acquires multimodal sensor data according to the sampling frequency of each sensor. S22. The sensor data acquisition process continuously writes sensor data through callback mechanism, publish-subscribe mechanism, shared cache, shared memory or message queue. The data generation process of different sensors does not need to be completed synchronously in the same simulation frame or the same physical moment. S23. Start the data preprocessing process for the vehicle intelligent agent, continuously read the latest available multimodal sensor data, and perform timestamp calibration, multi-source data alignment, coordinate system transformation, normalization, image preprocessing, point cloud preprocessing, vehicle status encoding and navigation command encoding on it to generate a unified input representation; S24. The data preprocessing process writes the unified input representation into shared memory, cache queue, publish-subscribe channel or inter-process communication channel for the autonomous driving algorithm execution process to read. S25. The vehicle intelligent agent initiates the execution process of the autonomous driving algorithm, which continuously reads the latest available unified input representation during operation and generates real-time driving decisions. S26. The vehicle intelligent agent starts the underlying vehicle control process. The underlying vehicle control process runs at high frequency as an independent asynchronous process. It continuously reads the latest available real-time driving decisions and generates vehicle control commands. S27. The sensor data acquisition process, data preprocessing process, autonomous driving algorithm execution process, and underlying vehicle control process run asynchronously as independent processes or threads, running units deployed on independent computing nodes, or a combination thereof. S28. The asynchronous operation mode of the independent process ensures that the simulation world does not stop progressing due to the time consumed by sensor data acquisition, data preprocessing, autonomous driving algorithm execution, or underlying vehicle control calculations, so that the end-to-end operation delay of the autonomous driving algorithm can truly affect the closed-loop driving behavior of the vehicle intelligent agent. S29. When the autonomous driving algorithm execution process fails to output a new real-time driving decision within a preset time, the underlying vehicle control process generates a vehicle control command using the most recent valid driving decision, trajectory extrapolation result, safety degradation strategy, smooth braking strategy, or lane keeping strategy. Specifically, the execution of the autonomous driving algorithm in step S25 includes: S251. Construct a pluggable autonomous driving algorithm access interface module, wherein the pluggable autonomous driving algorithm access interface module includes an input adaptation unit, an algorithm calling unit, and an output adaptation unit; S252, The input adaptation unit reads a unified input representation from shared memory, a cache queue, a publish-subscribe channel, or an inter-process communication channel, and converts it into the input format required by the autonomous driving algorithm to be evaluated; S253, The autonomous driving algorithm to be evaluated includes rule control algorithm, modular autonomous driving algorithm, end-to-end neural network model, imitation learning model, reinforcement learning model, online learning model, adaptive model, remote driving algorithm or a combination thereof; S254. The algorithm calling unit calls the autonomous driving algorithm to be evaluated based on the converted input format, so that it outputs real-time driving decisions. The real-time driving decisions include at least one of the following: target speed, future trajectory, waypoint sequence, steering angle, throttle command, braking command, lane change command, behavior decision, local path or low-level control command. S255. The output adaptation unit receives the real-time driving decisions output by the autonomous driving algorithm to be evaluated, and converts the decision results output by different algorithms into a unified driving decision format. When the unified driving decision format includes the target speed and the future trajectory, the underlying vehicle control process performs longitudinal control based on the target speed and lateral control based on the future trajectory. S256. The longitudinal control and lateral control are implemented using proportional-integral-derivative control, pure tracking control, model predictive control, vehicle dynamics control, or a combination thereof. S257. Continuously record the input timestamp, output timestamp, inference time, input data version, and output decision version of the autonomous driving algorithm to be evaluated for subsequent execution latency semantic monitoring and performance evaluation.

4. The asynchronous architecture-based closed-loop simulation and multi-dimensional online evaluation method for autonomous driving according to claim 3, characterized in that, Step S3 specifically involves: S31. Start the real-time status acquisition mechanism and establish a frame update callback, status subscription mechanism, event listening mechanism or a combination thereof driven by the simulation server or interactive environment; S32. In each state update cycle, parse the current simulation timestamp from the world snapshot object, state message or environment state object provided by the simulation server or interactive environment. S33. Obtain real-time status data of the vehicle intelligent agent, wherein the real-time status data includes at least one of real-time pose, velocity vector, acceleration vector, angular velocity, vehicle control status, vehicle bounding box, route progress, and road information where the vehicle is located. S34. Obtain real-time status data of background traffic participants, including background vehicles, pedestrians, static obstacles or other traffic participants; S35. Obtain traffic signal and road topology information, wherein the traffic signal and road topology information includes at least one of traffic signal light status, stop sign position, lane topology, road boundary, local speed limit, lane centerline and global route information; S36. Based on the predefined background traffic participant logical identifiers in the scenario configuration file, match them with the role name attribute or other object attributes set when the simulation vehicle is generated, and obtain the runtime object identifiers dynamically allocated by the simulation server, thereby establishing a mapping dictionary from logical identifiers to runtime object identifiers. S37. The mapping dictionary is used to quickly find background traffic participants in subsequent interaction harmony evaluation and continuously track their progress on preset routes, loop routes or task paths. S38. The acquired vehicle agent status, background traffic participant status, traffic signal status, road topology information, and simulation timestamp are integrated into a real-time status snapshot and transmitted to the asynchronous multidimensional online evaluation system.

5. The asynchronous architecture-based closed-loop simulation and multi-dimensional online evaluation method for autonomous driving according to claim 4, characterized in that, Step S3 also includes a pre-calculation process for traffic rule-related objects, specifically: S311, Instantiate the traffic light detection module, and scan all or part of the traffic light objects in the map after the simulation world is loaded; S312. Pre-calculate static geometric information for each traffic light object, wherein the static geometric information includes the traffic light trigger position, stop line waypoint, stop line vertex coordinates, and traffic light influence area. S313. The static geometric information of the traffic lights is cached in a static data structure or index structure for subsequent red light interaction detection and red light violation judgment. S314. Instantiate the stop sign detection module and filter candidate stop signs near the vehicle agent from the map object by using a distance threshold. S315. Iteratively generate a discrete path point sequence representing the future trajectory of the vehicle agent along the map waypoints. Determine whether the discrete path point sequence intersects with the two-dimensional projection area of ​​the stop sign trigger volume by point using the vector projection method. When the discrete path point sequence intersects with the two-dimensional projection area of ​​the stop sign trigger volume, determine that the corresponding stop sign is a path-related stop sign, and perform vehicle stop determination when the vehicle agent passes the stop sign.

6. The asynchronous architecture-based closed-loop simulation and multi-dimensional online evaluation method for autonomous driving according to claim 5, characterized in that, Step S4 specifically involves: S41. Start the asynchronous multidimensional online evaluation system and create an independent background evaluation process. The background evaluation process is decoupled from the simulation world's advancement process, the vehicle intelligent agent's asynchronous operation pipeline, and the autonomous driving algorithm's execution process. S42. The background evaluation process receives or reads the real-time state snapshot generated by the real-time state acquisition mechanism. The real-time state snapshot includes at least one of the following: vehicle agent state, background traffic participant state, traffic signal state, road topology information, vehicle control state, and simulation timestamp. S43. The background evaluation process performs online evaluation calculations according to the preset evaluation cycle, simulation frame update cycle, status snapshot arrival event or violation event triggering method, and updates the safety evaluation data, efficiency evaluation data, comfort evaluation data and interaction harmony evaluation data according to the real-time status snapshot. S44. The background evaluation process calculates safety evaluation indicators based on safety evaluation data, efficiency evaluation indicators based on route progress and speed limit compliance, comfort evaluation indicators based on vehicle dynamics smoothness and control continuity, and interaction harmony evaluation indicators based on the progress of background traffic participants. S45. The background evaluation process integrates the safety evaluation index, efficiency evaluation index, comfort evaluation index and interaction harmony evaluation index according to preset weights to obtain the comprehensive evaluation index at the current moment or the current evaluation window. S46. The background evaluation process writes the aforementioned safety evaluation index, efficiency evaluation index, comfort evaluation index, interaction harmony evaluation index, and comprehensive evaluation index into the long-term evaluation record according to the simulation timestamp or physical wall clock timestamp. S47. During the online evaluation calculation, the background evaluation process does not output control commands to the vehicle agent, does not participate in the simulation world time progression, and does not block sensor data acquisition, data preprocessing, autonomous driving algorithm execution, and underlying vehicle control processes.

7. The asynchronous architecture-based closed-loop simulation and multi-dimensional online evaluation method for autonomous driving according to claim 6, characterized in that, In step S4, the calculation process of the security evaluation index is as follows: S411. In the quantitative assessment of safety, the safety evaluation is divided into four sub-indicators: collision detection, emergency braking detection, lane correctness detection, and traffic rule detection. S412. In collision detection, a collision sensor is added to the vehicle intelligent agent and a collision event callback function is registered to monitor collision events in real time. A time window filtering method based on collision objects is adopted to maintain a latest collision timestamp for each independent object that collides. When a new collision event is detected, it is only regarded as a new independent collision event if the difference between the timestamp of the event and the latest collision timestamp recorded by the corresponding object exceeds a preset time threshold. The identified independent collision events are classified and stored according to the collision object type, which includes pedestrians, vehicles and static objects. S413. In emergency braking detection, the acceleration vector of the vehicle agent in the world coordinate system is obtained and transformed to the vehicle's own local coordinate system to obtain the longitudinal acceleration. When the current speed of the vehicle agent is higher than the preset high speed threshold and the longitudinal acceleration in its local coordinate system is lower than the preset severe deceleration threshold, an emergency braking candidate event is determined to have occurred. The emergency braking candidate event is filtered by a time window method. The current emergency braking event is confirmed and recorded only when the time since the last validly recorded emergency braking event exceeds the preset time window. S414. In lane correctness detection, multiple bottom corner points of the vehicle agent bounding box are sampled, and the map waypoints below these sampled points are queried and verified to determine whether the vehicle agent is driving beyond the lane type. The dot product between the forward vector of the vehicle agent and the forward vector of its lane or projected waypoint is calculated to determine whether it is driving in reverse. If the dot product is less than a preset threshold, the vehicle agent is determined to be driving in reverse. Calculate the shortest Euclidean distance between the current position of the vehicle agent and the predetermined global route. If the distance exceeds the preset route deviation threshold, it is determined that a route deviation has occurred. S415. In traffic rule detection, a state machine-based delayed penalty method is adopted, which delays the violation judgment time from the moment the vehicle behavior occurs until after the vehicle has completely left the area affected by the corresponding traffic rule; during the interaction with traffic lights, when the front end of the vehicle's intelligent agent first reaches the stop line, if the traffic light is detected to be red, the state machine is set to a potential violation state. The vehicle agent's position is then continuously monitored until it completely leaves the traffic light trigger area. If the traffic light is still red and the state machine was previously in a potential violation state at the moment the vehicle agent leaves, a red light violation event is confirmed and recorded. During the interaction with the stop sign, when the vehicle agent enters the stop sign trigger area, the state machine begins to monitor the vehicle speed to determine whether a valid stopping behavior has occurred. When the vehicle agent is detected to have completely left the trigger area, if no valid parking behavior has occurred during the entire interaction process, an illegal parking event will be confirmed and recorded. S416. Treat collision events and emergency braking events as discrete risk events, and cross-border driving, driving against traffic, and route deviation as persistent violations. Calculate the occurrence rate per kilometer of discrete risk events and the time percentage of persistent violations, and calculate the basic safety score based on a cumulative penalty method. : in, Discrete risk events The penalty factor Discrete risk events The number of times it occurs, This represents the total mileage traveled by the vehicle. For persistent violations The percentage of time spent on this topic; S417. Calculate the total number of interactions with traffic rules and the number of violations, and calculate traffic rule compliance points. : in, For traffic rules Total number of interaction opportunities For traffic rules The number of violations; S418. The basic safety score and the traffic rule compliance score are weighted and summed to obtain the final safety score. As a safety evaluation indicator: in, Preset security fusion weights; The specific calculation process for the efficiency evaluation index is as follows: S421. During the efficiency evaluation process, the current position of the vehicle agent is projected onto the predetermined global route to find the point on the route closest to the vehicle, thereby determining its effective travel distance on the path. , used to characterize the actual degree of progress of the vehicle agent along the predetermined global route; S422, Effective forward distance With total path length The ratio is converted to route completion. : S423. During the evaluation process, the local speed limit of the lane currently occupied by the vehicle agent is continuously obtained through the map interface, road topology interface, or simulation server interface. And obtain the current real-time speed of the vehicle's intelligent agent. Instantaneous velocity score is calculated using a nonlinear penalty function. To punish speeding: S424, For all sampling times The instantaneous speed scores are averaged and converted to a percentage to obtain the average speed score. : in, The number of sampling frames during the evaluation process; S425. Normalize the route completion score and average speed score, then multiply them to obtain the efficiency score. As an efficiency evaluation indicator: The specific calculation process for the comfort evaluation index is as follows: S431. Maintain a fixed-length first-in-first-out queue for the acceleration vector, throttle control quantity, braking control quantity and steering control quantity of the vehicle intelligent agent respectively. By calculating the arithmetic mean of the data in the first-in-first-out queue, perform moving average smoothing on the acceleration vector, throttle control quantity, braking control quantity and steering control quantity. S432. Based on the smoothed data, calculate the original indicators for comfort evaluation. These original indicators include the root mean square of longitudinal acceleration, the root mean square of lateral acceleration, the root mean square of the steering rate of change, and the frequency of alternation between throttle and brake commands. The longitudinal acceleration... The following is obtained by approximating the smoothed longitudinal acceleration with respect to time: S433, Raw comfort index at any given time its root mean square Calculated using the following formula: S434, each original comfort index Input a nonlinear scoring function based on exponential decay to obtain the scores for the corresponding sub-items. : in, This is the severity coefficient. This is the ideal benchmark value for the corresponding indicator; S435. Scoring multiple comfort sub-items based on preset weights. A weighted average is used to obtain the comfort score. : in, For the number of comfort sub-items, For the first The weights of each comfort sub-item are assigned, and the sum of the weights of each comfort sub-item is 1. The specific calculation process for the interaction harmony evaluation index is as follows: S441. During the evaluation process, the interaction harmony is calculated by quantifying the impact of the vehicle intelligent agent on the efficiency of background traffic flow. S442. The system tracks the progress of each background traffic participant in real time on their preset path, loop path, or task path; for the first... The task completion rate of each background traffic participant is calculated using the waypoint arrival method. : in, It is the first Background traffic participants at time Number of laps completed Index its current destination waypoint. The total number of waypoints along its path; S443. Calculate the arithmetic mean of the task completion rates of all background traffic participants to obtain the average traffic flow completion rate at the current moment. : in, The number of traffic participants in the background; S444. Store the average completion rate of traffic flow along with the current timestamp into the traffic flow time series. At the end of the evaluation, perform a discounted numerical integration on the traffic flow time series to obtain the actual discounted cumulative completion rate. : in, As a time discount factor, To assess the start time; S445. Based on the reference completion rate, the expert system's ideal traffic flow model, or the preset ideal traffic model, calculate the cumulative completion rate of the ideal discount according to the actual timestamp sequence of this assessment. First, scene calibration is performed under conditions free from interference from the tested vehicle's intelligent agent. This ensures that the background traffic participants, test route, traffic signals, road topology, and traffic management policies are consistent with the formal evaluation scenario. The average route completion rate of the background traffic participants over time during the calibration process is statistically analyzed, and the average reference completion rate of the background traffic participants is calculated accordingly. And if this is used as a reference completion rate under ideal conditions, then the ideal reference completion curve is: The ideal discount cumulative completion rate Calculate using the following formula: in, To assess the start time, and For adjacent sampling timestamps, This represents the number of sampling points in the traffic flow time series. Time discount factor; S446. Calculate the interaction harmony score based on the actual cumulative discount completion rate and the ideal cumulative discount completion rate. : S447. The interaction harmony score is used as an interaction harmony evaluation index to characterize the impact of the vehicle agent on the background traffic flow; when the vehicle agent causes background traffic participants to frequently slow down, stop, detour, or experience a decrease in traffic progress, the interaction harmony score decreases.

8. The asynchronous architecture-based closed-loop simulation and multi-dimensional online evaluation method for autonomous driving according to claim 7, characterized in that, Step S5 specifically involves: S51. During the operation of the vehicle intelligent agent, the end-to-end physical wall clock delay is recorded. The end-to-end physical wall clock delay represents the actual time consumed on the physical hardware between the generation of sensor data and the generation of vehicle control commands. S52. During the same end-to-end execution process, record the simulation elapsed delay, whereby the simulation elapsed delay represents the simulation time elapsed within the simulation world during the end-to-end execution process. S53. Calculate the delay consistency ratio, which is the ratio between the simulated elapsed delay and the end-to-end physical wall clock delay. The formula is as follows: in, This represents the latency consistency ratio. To simulate elapsed delay, End-to-end physical wall clock delay; S54. Calculate the real-time factor of the simulation system. The real-time factor represents the ratio between the simulation time elapsed in the simulation world and the physical wall clock time in a non-blocking, non-freezing state. It is used to characterize the speed at which the simulation world progresses relative to the physical wall clock time during asynchronous closed-loop interaction. The calculation formula is as follows: in, For the real-time factors of the simulation system, and These are the simulation times at the beginning and end of the evaluation window, respectively. and These are the physical wall clock times at the start and end of the evaluation window, respectively; S55. When the time delay consistency ratio is equal to or close to the real-time factor, it is determined that the actual computational delay of the autonomous driving algorithm to be evaluated is mapped to the closed-loop dynamics. S56. When the time delay consistency ratio is close to zero, it is determined that the simulation world has time freeze semantics during the algorithm operation, that is, the algorithm calculation delay has not entered the closed-loop dynamics.

9. The asynchronous architecture-based closed-loop simulation and multi-dimensional online evaluation method for autonomous driving according to claim 8, characterized in that, The specific process of S6 is as follows: S61. During the long-term closed-loop interaction process, the background evaluation process continuously records scores for each dimension according to timestamps. These scores include security scores, efficiency scores, comfort scores, and interaction harmony scores. A comprehensive score is calculated based on these scores. Calculate according to the following formula: in, , , and The weights are assigned to safety, efficiency, comfort, and interaction harmony, respectively, and the sum of each weight is 1. S62. The background evaluation process continuously records specific violations and the frequency of violations per unit distance. The specific violations include at least one of pedestrian collisions, vehicle collisions, static object collisions, emergency braking, leaving the effective lane, driving in the wrong direction, deviating from the route, running a red light, and improper parking. The frequency of violations per unit distance includes the frequency of occurrence of various collision events and emergency braking events per kilometer. S63. The background evaluation process continuously records the background traffic flow time series, which includes the task completion rate of each background traffic participant, the average completion rate of the traffic flow, the timestamp, and the intermediate amount of the discount points. S64. The background evaluation process continuously records latency semantic data, which includes end-to-end physical wall clock latency, simulated elapsed latency, latency consistency ratio, simulated frame rate, and real-time factor. S65. The background evaluation process updates the above-mentioned recorded data according to fixed time intervals, fixed frame intervals or event triggering methods, and performs statistical summarization on the comprehensive score, scores of each dimension, violation frequency per unit distance, background traffic flow time series and time delay semantic data to generate real-time evaluation curves, phased evaluation results or statistical results for report generation. S66. Determine whether the long-range closed-loop interaction process meets the termination conditions. When at least one of the following termination conditions is met: fixed physical wall clock time window, fixed simulation time window, fixed route task, or fixed number of interaction events, terminate the long-range closed-loop interaction process. S67. After the evaluation process is completed, the system generates a comprehensive performance evaluation report based on the recorded data and statistical results; S68. The system generates a visual evaluation curve, algorithm comparison table, or phased performance summary based on the comprehensive performance evaluation report to support the horizontal comparison of different autonomous driving algorithms under the same asynchronous closed-loop simulation environment.

10. An autonomous driving closed-loop simulation and multi-dimensional online evaluation system based on asynchronous architecture, characterized in that, include: The module includes the establishment and configuration of the asynchronous autonomous driving simulation environment, the evaluation index calculation module, the judgment module for whether the autonomous driving algorithm and the closed-loop dynamics are mapped, and the autonomous driving performance evaluation report generation module. The asynchronous autonomous driving simulation environment establishment and configuration module can implement the contents of steps S1 to S3 as described in any one of claims 1-9; the evaluation index calculation module can implement the contents of step S4 as described in any one of claims 1-9; the autonomous driving algorithm and closed-loop dynamics mapping determination module can implement the contents of step S5 as described in any one of claims 1-9; and the autonomous driving performance evaluation report generation module can implement the contents of step S6 as described in any one of claims 1-9.

Images