A method and apparatus for safety evaluation of autonomous driving based on a world model
By constructing a high-fidelity closed-loop simulation environment through a world-model-based safety evaluation method, and combining a multimodal prompting lexicon and a reinforcement learning prompter to generate complex long-tailed scenarios, the problem of low efficiency and large simulation gap in autonomous driving safety evaluation is solved, achieving efficient and accurate safety evaluation results.
Patent Information
- Application Number
- CN202611352012.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-09-02
- Publication Date
- 2026-11-17
AI Technical Summary
Existing autonomous driving safety evaluation technologies struggle to improve evaluation efficiency while ensuring scene fidelity, especially when generating complex long-tailed scenes, which suffer from sparsity curse and the gap between simulation and reality.
A world-model-based safety evaluation method is adopted. A high-fidelity closed-loop simulation environment is constructed by generative driving world model. Complex long-tail scenarios are generated by combining multimodal prompting word library and reinforcement learning prompter, and comprehensive evaluation is carried out by multi-dimensional discriminator.
It enables efficient and accurate safety evaluation of next-generation autonomous driving algorithms, solving the problems of low efficiency and large gap between simulation and reality in traditional methods. It can generate high-fidelity, controllable complex scenarios and provide standardized safety verification.
Smart Images

Figure CN123116831A_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of autonomous driving safety evaluation technology, specifically relating to an autonomous driving safety evaluation method and device based on a world model. Background Technology
[0002] With the deep integration of artificial intelligence and the automotive industry, autonomous driving technology is at a critical juncture, transitioning from research and development to large-scale commercial deployment. Throughout this process, safety remains the primary objective of autonomous driving systems, and safety evaluation is the crucial final step in determining whether autonomous vehicles can truly enter everyday life. One of the main driving forces behind the development and adoption of autonomous vehicles is improving road safety, which can typically be quantified as the degree and frequency of traffic violations or accidents. To accurately derive this quantitative indicator, driving safety evaluation requires long-term statistical analysis and validation of autonomous driving systems in extremely realistic and complex environments. However, the development of current autonomous driving algorithms has reached a bottleneck: traditional algorithmic paradigms have been exhausted, performance metrics on various publicly available benchmarks are nearing saturation, but "long-tail data" that truly reflects extremely dangerous situations is extremely scarce. This imbalance in data distribution poses a severe challenge to existing safety evaluation systems.
[0003] Traditional autonomous driving safety evaluation technologies struggle to effectively balance the two core metrics of "evaluation efficiency" and "scenario fidelity" in practical applications. Specifically, driving safety evaluation faces two fundamental challenges: the "curse of sparsity" and "low scenario fidelity." The "curse of sparsity," also known as the "billion-mile problem," stems from the inherent sparsity of safety-critical scenarios in the real world (i.e., dangerous scenarios highly prone to accidents). Statistics show that to prove an autonomous driving system achieves a collision rate of 1 collision per million miles (i.e., exceeding the safety level of human drivers), at least 8.8 billion miles of real-world testing would be required, which is completely unacceptable in terms of both time and economic costs. On the other hand, "scenario fidelity" determines whether the evaluation results truly reflect the system's driving performance in the real physical world. If the test environment differs significantly from the real world, a system that performs exceptionally well in testing may still lead to serious safety accidents on real roads due to insufficient algorithm robustness.
[0004] Looking at existing driving safety assessment technologies, the first three generations all have significant limitations. The first generation, represented by traditional simulators (such as Waymo CarCraft) and closed-road testing (such as Waymo Castle), is limited by traditional graphics rendering and finite physical rules, resulting in low scene fidelity and low testing efficiency. The second generation, represented by real-vehicle road testing (such as Tesla and Waymo's actual road tests), while offering extremely high scene fidelity, suffers from the "curse of sparsity," failing to efficiently acquire long-tail hazardous scenarios and maintaining low assessment efficiency. The third generation introduces dense reinforcement learning (such as RoboSafe) for test generation, which improves assessment efficiency to some extent, but the generated scenes often deviate from real physical and visual laws, and scene fidelity still falls short of requirements. Furthermore, with the emergence of next-generation autonomous driving algorithms such as end-to-end, vision-language-action (VLA) models, and world agents (WA), these algorithms heavily rely on high-dimensional real sensor input (such as multi-view video). Traditional simulators suffer from a huge "sim-to-real gap," making it impossible to provide high-fidelity visual and physical feedback for these advanced algorithms, and even more difficult to achieve efficient and controllable generation of complex long-tailed scenes.
[0005] Therefore, how to build a high-fidelity and highly controllable simulation platform to break through the constraints of traditional evaluation technology in terms of efficiency and realism, and at the same time solve the problems of "curse of sparsity" and "low scene fidelity", so as to meet the safety evaluation needs of the new generation of autonomous driving models in complex long-tail scenarios, has become a key technical problem that urgently needs to be solved. Summary of the Invention
[0006] The present invention aims to at least partially solve one of the technical problems in the related art.
[0007] Therefore, the first objective of this invention is to propose a safety evaluation method for autonomous driving based on a world model.
[0008] The second objective of this invention is to propose an autonomous driving safety evaluation device based on a world model.
[0009] The third objective of this invention is to provide a computer device.
[0010] A fourth objective of this invention is to provide a non-transitory computer-readable storage medium.
[0011] To achieve the above objectives, a first aspect of the present invention proposes an autonomous driving safety evaluation method based on a world model, comprising: High-fidelity multi-view video frames and environmental states are generated using a driving world model, which internalizes the distribution of real driving data into the model in a generative manner. Based on a multimodal cue lexicon and a reinforcement learning cue, a targeted generation of safety-critical long-tail scenarios covering simple to difficult scenarios is achieved, and a traffic flow simulator is driven to generate dynamic trajectories that conform to physical laws. The autonomous driving model under test is connected to the closed-loop simulation and its output control actions are received. The driving world model generates multi-view video frames and environmental states for the next moment based on the current environmental state and control actions, forming a closed-loop interaction of perception-decision-environmental feedback. The system uses a multi-dimensional discriminator to calculate collision risk, traffic rule violations, driving comfort, and driving efficiency in real time, and outputs a comprehensive safety assessment result.
[0012] In one embodiment of the present invention, constructing the driving world model includes: The latent space video diffusion network is trained based on real road multi-view videos, vehicle motion and control records, traffic participant trajectories, high-precision maps, traffic signal status, and scene text descriptions. The training includes visual representation and video generation pre-training based on real videos, controllable generation training with multimodal conditions, causal closed-loop training with vehicle actions and historical states, and joint consistency training using cross-view geometric correspondence, instance identity preservation, and temporal continuity loss.
[0013] In one embodiment of the present invention, the visual representation and video generation pre-training based on real video includes: The continuous multi-view camera images are compressed into spatiotemporal latent variables by a video encoder. The spatiotemporal causal generation backbone predicts the latent variables of the next time step based on historical latent variables, current environmental state and vehicle control actions. The state prediction head synchronously outputs the structured states of vehicles, traffic participants, and road elements, and the video decoder restores the predicted latent variables into multi-view video frames.
[0014] In one embodiment of the present invention, the reinforcement learning cue is modeled as a Markov decision process, and its state space includes at least one or more of the following: road topology, motion states of the vehicle and background traffic participants, current multimodal cue features, and historical actions of the algorithm under test; its action space includes adjusting the motion parameters of the background traffic participants and selecting or perturbing the multimodal cue conditions; and the single-step reward is represented as a weighted combination of risk reward, authenticity reward, diversity reward, and physical constraint penalty.
[0015] In one embodiment of the present invention, the closed-loop interaction is executed sequentially as follows: reading the video and status of the previous moment, calling the algorithm under test to obtain the control action, verifying the action format and range, advancing the traffic flow status, calling the driving world model to generate the observation of the next moment, calculating the discrimination index and writing it into the log; the interaction frequency is configured according to the planning cycle of the algorithm under test.
[0016] In one embodiment of the present invention, the multi-dimensional discriminator includes: The collision discriminator is used to calculate the spatial overlap between the vehicle detection box and other target detection boxes at each time step. When the overlap exceeds a preset threshold, it is recorded as a collision. The traffic rule discriminator is used to project the vehicle's location and trajectory onto the road topology and generate violation events based on lane boundaries, center lines, driving directions, stop lines, drivable areas, and traffic light phases. The human factor discriminator is used to calculate longitudinal and lateral acceleration and jerk based on the vehicle's speed sequence to form a comfort index.
[0017] In one embodiment of the present invention, the comprehensive safety assessment result is obtained by weighted sum of collision risk, traffic regulation risk, comfort risk and efficiency risk, with the weights fixed in the same assessment task and saved with the report; the system simultaneously outputs time-step raw indicators, scenario-level individual indicators, comprehensive results, event timestamps and evidence fragments.
[0018] To achieve the above objectives, a second aspect of the present invention provides an autonomous driving safety evaluation device based on a world model, comprising: The world model generation module is used to generate high-fidelity multi-view video frames and environmental states using a driving world model. The driving world model internalizes the distribution of real driving data into the model in a generative manner. The long-tail scene generation module is used to generate safety-critical long-tail scenes ranging from simple to difficult based on a multimodal cue lexicon and a reinforcement learning cue, and to drive the traffic flow simulator to generate dynamic trajectories that conform to physical laws. The closed-loop simulation interaction module is used to connect the autonomous driving model under test to the closed-loop simulation, receive the control actions output by it, and generate multi-view video frames and environmental states for the next moment by the driving world model based on the current environmental state and control actions, forming a closed-loop interaction of perception-decision-environmental feedback. The multi-dimensional evaluation module is used to calculate collision risk, traffic rule violations, driving comfort and driving efficiency indicators in real time through a multi-dimensional discriminator, and output a comprehensive safety evaluation result.
[0019] This invention discloses a world-model-based autonomous driving safety evaluation method and apparatus. By constructing a new generation of autonomous driving safety evaluation system based on a world model, and integrating multimodal prompts and high-fidelity closed-loop simulation, it achieves efficient and accurate safety evaluation for next-generation autonomous driving algorithms. Compared with existing technologies, this invention eliminates the "simulation-to-reality gap" (S2R Gap) in traditional simulation testing by introducing a generative driving world model to construct a digital twin. While ensuring high fidelity, it achieves accurate response to complex control signals and long-term generation from multiple perspectives. Simultaneously, by combining a massive multimodal prompt library and reinforcement learning prompters, the system can generate various long-tailed hazardous scenarios in a targeted and efficient manner, effectively solving the "curse of sparsity" and the "billion-mile problem" faced by traditional real-vehicle testing. Furthermore, by combining a comprehensive quantitative evaluation with a multi-dimensional discriminator and a difficult example mining mechanism, the system can not only provide standardized safety verification for cutting-edge models such as end-to-end and VLA, but also use the generated proprietary data to feed back into the autonomous driving model, thereby significantly accelerating the safe deployment and large-scale application of autonomous driving technology.
[0020] To achieve the above objectives, a third aspect of this application provides a computer device, including a processor and a memory; wherein the processor reads executable program code stored in the memory to run a program corresponding to the executable program code, for implementing an autonomous driving safety evaluation method based on a world model as described in the first aspect embodiment.
[0021] To achieve the above objectives, the fourth aspect of this application proposes a non-transitory computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements an autonomous driving safety evaluation method based on a world model as described in the first aspect embodiment.
[0022] Additional aspects and advantages of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. Attached Figure Description
[0023] The above and / or additional aspects and advantages of the present invention will become apparent and readily understood from the following description of the embodiments taken in conjunction with the accompanying drawings, wherein: Figure 1 This is a flowchart of an autonomous driving safety evaluation method based on a world model according to an embodiment of the present invention; Figure 2 This is a structural diagram of an autonomous driving safety evaluation system based on a world model according to an embodiment of the present invention; Figure 3 This is a modular disassembly diagram of a driving safety evaluation platform according to an embodiment of the present invention; Figure 4This is a comparison chart of the results of the embodiments of the present invention and the control scheme; Figure 5 This is a structural diagram of an autonomous driving safety evaluation device based on a world model according to an embodiment of the present invention; Figure 6 It is a computer device according to an embodiment of the present invention. Detailed Implementation
[0024] It should be noted that, unless otherwise specified, the embodiments and features described in the present invention can be combined with each other. The present invention will now be described in detail with reference to the accompanying drawings and embodiments.
[0025] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.
[0026] The following description, with reference to the accompanying drawings, describes an autonomous driving safety evaluation method and apparatus based on a world model according to an embodiment of the present invention.
[0027] Example 1 Figure 1 This is a flowchart of an autonomous driving safety evaluation method based on a world model according to an embodiment of the present invention, such as... Figure 1 As shown, it includes: S1, High-fidelity multi-view video frames and environmental states are generated using a driving world model. The driving world model internalizes the distribution of real driving data into the model in a generative manner. S2, based on a multimodal prompting lexicon and a reinforcement learning prompter, generates targeted long-tail safety-critical scenarios covering simple to difficult scenarios, and drives a traffic flow simulator to generate dynamic trajectories that conform to physical laws. S3, the autonomous driving model under test is connected to the closed-loop simulation, and the control actions output by it are received. The driving world model generates multi-view video frames and environmental states for the next moment based on the current environmental state and control actions, forming a closed-loop interaction of perception-decision-environmental feedback. S4 uses a multi-dimensional discriminator to calculate collision risk, traffic rule violations, driving comfort, and driving efficiency in real time, and outputs a comprehensive safety assessment result.
[0028] The present invention provides an autonomous driving safety evaluation method based on a world model, which can effectively improve the efficiency and scene fidelity of autonomous driving safety evaluation, and solve the problems of low evaluation efficiency and large gap between simulation and reality in traditional methods.
[0029] The following describes in detail an autonomous driving safety evaluation method based on a world model according to an embodiment of the present invention, with reference to the accompanying drawings.
[0030] 1. A high-fidelity closed-loop simulation method based on a driving world model.
[0031] To address the "simulation-to-reality gap" (S2R Gap) problem in traditional simulation testing, this invention proposes a high-fidelity closed-loop simulation technology based on a generative driving world model, providing a digital twin testing environment of real physical scenarios for autonomous driving algorithms.
[0032] In one implementation, the driving world model employs a conditional temporal prediction architecture with a latent space video diffusion network or equivalent generative network as its backbone. This architecture includes at least a video encoder, a spatiotemporal causal generative backbone, a multimodal conditional encoder, an action condition injection module, an environmental state prediction head, a multi-view consistency constraint module, and a video decoder. The video encoder compresses continuous multi-view camera images into spatiotemporal latent variables; the generative backbone accesses only current and historical time-stamp information and predicts the latent variables for the next time step based on historical latent variables, the current environmental state, and the vehicle's control actions; the state prediction head synchronously outputs the structured states of vehicles, traffic participants, and road elements; and the video decoder restores the predicted latent variables into multi-view video frames for use by the algorithm under test. The network type described above is not limited to a specific product or model; diffusion networks, generative transformers, or other generative networks capable of conditional temporal generation and action response can be used as alternative implementations.
[0033] The model training data consists of time-synchronized real-world multi-view videos of roads, vehicle motion and control records, traffic participant trajectories, high-precision maps or road topology, traffic signal status, and scene text descriptions. The training process can sequentially include: pre-training based on visual representations and video generation from real-world videos; controlled generation training incorporating text, reference images, detection boxes, maps, and trajectory conditions; causal closed-loop training incorporating vehicle actions and historical states; consistency training utilizing cross-view geometric correspondence, instance identity preservation, and temporal continuity loss; and reducing inference latency through causal distillation and few-step distillation. In one embodiment, the number of sampling steps is reduced from 50 steps to 4 steps, a reduction of over 90%.
[0034] During inference, the system takes the multi-view video, environmental state, multimodal scene conditions, and control actions output by the algorithm under test from the most recent time steps as input. Through conditional encoding and spatiotemporal causal generation, the latent variables and structured state of the next time step are obtained, and then decoded into the next set of multi-view video frames. The generated video frames and states are fed back to the algorithm under test on the one hand, and sent to the discriminator to calculate the evaluation index on the other hand, thus forming a closed loop.
[0035] First, this invention constructs a dynamic interactive closed loop between a world model simulator and the autonomous driving algorithm under test. At each time step of the closed-loop simulation... The world model simulator, acting as the environment provider, outputs high-fidelity multi-view video frames and environmental states to the autonomous driving model under test (such as an end-to-end model, VLA model, or world agent WA). The algorithm under test receives and processes this high-dimensional sensor data, performs perception and planning decisions, and outputs the vehicle's control actions for the current moment. (Such as control signals for steering angle, acceleration, etc.). The world model simulator then receives this action. In conjunction with the current environmental context, it autoregressively infers and generates the next time step. The system uses multi-view video frames and environmental states. This process continues at a specific interaction frequency (e.g., 1Hz), forming a complete "perception-decision-environmental feedback" closed loop, realistically reproducing the vehicle's continuous driving and interaction process on the physical road. The system structure is shown in the attached figure. Figure 2 As shown.
[0036] The closed-loop interface can transmit data packets with a unified timestamp using memory messages, inter-process communication, remote procedure calls, or other equivalent methods. Video frame data can be organized as V[t,v,h,w,c], where t is the time index, v is the camera viewpoint, h and w are the image dimensions, and c is the color or sensor channel; the number of viewpoints, resolution, frame rate, and time window are configured according to the interface of the algorithm under test. For example, one or more viewpoints from the front, rear, left, right, and surround-view cameras can be output, with images in RGB format and carrying camera intrinsic and extrinsic parameters, frame number, and acquisition timestamp.
[0037] Understandably, environmental state data should include at least: vehicle position, attitude, and historical trajectory; category, position, orientation, speed, 3D bounding box, and identifier of each traffic participant; and road topology such as lane centerline, lane boundaries, drivable area, and intersection connections. Control action data should include at least steering angle or equivalent curvature, longitudinal acceleration, or throttle and braking amounts; steering angle can be expressed in radians or degrees, longitudinal acceleration in meters per second squared, and throttle and braking amounts can be normalized to [0,1]. Each action data packet carries an action activation timestamp and duration to align with video frames and environmental states. Evaluation metrics data include single-step collision, violation, comfort, and efficiency metrics, as well as scene-level cumulative values.
[0038] A closed-loop interaction executes sequentially: reading the video and status of the previous moment, calling the algorithm under test to obtain actions, verifying the action format and range, advancing traffic flow status, calling the world model to generate the next moment's observation, calculating the discrimination index and writing it to the log. The interaction frequency can be configured from 0.5 Hz to 20 Hz or other frequencies according to the planning cycle of the algorithm under test; 1 Hz is only one example. When frame loss, timestamp exceeding tolerance, numerical out-of-bounds, or generation timeout occurs, the system can adopt methods such as maintaining the previous valid value, interpolation, action limiting, reducing the interaction frequency, retrying, or terminating the current scene, and record the abnormal flag to prevent abnormal samples from being included in the normal evaluation results.
[0039] Secondly, this driving world model possesses extremely high controllability, strict causality, low latency, and powerful multi-view generation capabilities. Regarding controllability, the world model supports joint input of multimodal conditions, including textual conditions (such as the natural language description of a "busy intersection"), image conditions (such as specific weather and lighting backgrounds), and structured conditions (such as high-precision map topology, instance detection boxes, and ground truth trajectories). Among these, textual conditions are derived from video features. Text features extracted by language models Injection of cross-attention mechanisms between:
[0040] Image conditions are achieved by using image features and video features After being concatenated along the temporal dimension, it is then injected through a self-attention mechanism:
[0041] The structured conditions are obtained by projecting detection boxes and map elements onto the camera's viewpoint corresponding to the video, thus generating two-dimensional features, which are then concatenated with video features along the channel dimension. Based on these conditions, the model can not only generate specific scenes in a targeted manner but also accurately respond to complex control signals output by the algorithm under test (such as driving on an S-curve), achieving fine-grained control at both the instance and map levels. Regarding causality and generative capability, the model employs a spatiotemporal causal attention mechanism to ensure that the generated video sequences strictly adhere to physical laws and temporal coherence, avoiding the spatiotemporal distortion problems common in traditional generative models.
[0042] Meanwhile, the model undergoes minimal distillation, resulting in extremely low inference latency, perfectly meeting the interactive requirements of closed-loop simulation. Furthermore, the model supports the generation of long-duration (e.g., over 20 seconds) and multi-view high-fidelity continuous video, providing comprehensive environmental observation for advanced autonomous driving algorithms that heavily rely on multi-dimensional visual input, thus ensuring the authenticity and reliability of safety evaluation results.
[0043] 2. A method for generating long-tail scenes based on reinforcement learning prompts.
[0044] To address the "billion-mile problem" caused by the extreme sparseness of safety-critical scenarios in the real physical world, this invention proposes a long-tail scene generation method based on a multimodal cue lexicon and a reinforcement learning cue, enabling efficient and targeted mining of complex and dangerous edge scenarios. The operation of this cue within the overall evaluation framework is illustrated in the appendix. Figure 3 As shown.
[0045] First, this invention constructs an instance-level traffic flow simulator. Based on a massive multimodal cue word library, the system extracts structured cue information such as high-precision map topology and the initial spatial arrangement of traffic participants (e.g., vehicles, pedestrians). Using this as a foundation, the instance-level traffic flow simulator generates dynamic trajectories that conform to physical laws for each background traffic instance in the scene. This instance-level fine-grained control not only provides high-fidelity structured conditional input for the driving world model but also ensures the realism and diversity of complex traffic interactions.
[0046] Secondly, this invention introduces reinforcement learning technology to fine-tune safety-critical scenarios and mine challenging examples. During scenario generation and closed-loop interaction, the reinforcement learning algorithm uses the safety performance of the autonomous driving model under test (such as collision risk and violation probability) and the current state of the scenario as feedback signals. When the generated traffic flow poses a higher risk to the model under test, the agent receives a higher reward. Through this adversarial reinforcement learning mechanism, the system can continuously fine-tune the behavioral parameters (such as aggression level and acceleration / deceleration timing) or cue word features of each instance in the traffic flow simulator, proactively creating highly challenging safety-critical scenarios (such as sudden crossings and extreme lane-changing). This method effectively overcomes the distribution limitations of conventional data, achieving massive generation of long-tail hazardous scenarios and spatiotemporal data completion.
[0047] The reinforcement learning cue can be modeled as a Markov decision process. The state space includes at least the road topology, the motion states of the vehicle and background traffic participants, the current multimodal cue features, and the historical actions of the algorithm under test. The action space includes continuous or discrete adjustments to the target speed, acceleration / deceleration timing, lane-changing intention, crossing timing, following distance, and aggressiveness of background traffic participants, as well as selection or perturbation of cue conditions such as text, reference images, detection boxes, map elements, and target trajectories. The policy network can employ Actor-Critic, proximal policy optimization, or other reinforcement learning networks capable of handling continuous / discrete actions; the specific algorithm is not limited.
[0048] In one implementation, the single-step reward is represented as a weighted combination of risk reward, authenticity reward, diversity reward, and physical constraint penalty. The risk reward increases with collisions, violations, reduced expected collision times, or decreased minimum safe distances; the authenticity reward is determined based on the likelihood, speed, and acceleration reasonableness of the generated trajectory relative to the distribution of real driving data; the diversity reward is used to reduce redundancy with saved scenarios in terms of road structure, participant behavior, and cue features; and the physical constraint penalty is used to suppress situations such as vehicle teleportation, detection box penetration, exceeding road boundaries, and exceeding preset dynamic ranges. The weights can be calibrated based on a validation set and stored in the evaluation configuration, allowing different algorithms under test to be compared horizontally using the same set of configurations.
[0049] Each training round samples an initial scenario from the cue word library. The traffic flow simulator interacts in a closed loop with the algorithm under test. The discriminator calculates the reward, and the cue reader updates its strategy accordingly. A round terminates when a collision occurs, the objective is reached, the maximum simulation time is exceeded, multiple consecutive steps violate physical constraints, or the generation module malfunctions. Scenarios that meet the risk threshold and pass the realism and physical constraint verification are marked as difficult examples and written into the scenario replay library along with a random seed, initial state, cue conditions, action sequence, and evaluation results. After deduplication and manual or automatic review, the cue word library is refilled for subsequent reproduction, regression testing, and model training.
[0050] 3. Unified driving safety evaluation indicators that include multi-dimensional discriminators.
[0051] To comprehensively and objectively quantify and evaluate the overall performance of autonomous driving algorithms in complex, long-tail scenarios, this invention constructs a unified driving safety evaluation index system that includes a multi-dimensional discriminator (Judger). This system collects real-time data on the vehicle's state and its interaction with the environment during closed-loop simulation, and performs precise evaluations from three core dimensions. The operation of this evaluation index within the overall evaluation framework is shown in the attached figure. Figure 3 As shown.
[0052] First, there are traffic rule-related indicators. The system has a built-in traffic rule discriminator that, combined with high-precision map topology and environmental information such as traffic signal status, monitors in real time whether the vehicle strictly adheres to traffic regulations while driving. Specific evaluation indicators include, but are not limited to, the frequency and severity of violations such as crossing solid lines, running red lights, driving in the wrong direction, failing to yield as required, and deviating from lanes, thereby assessing the compliance of the algorithm.
[0053] Secondly, there are indicators related to driving risk. Safety is the primary goal of autonomous driving, and the system rigorously quantifies driving risk through a collision discriminator. This discriminator calculates in real time the spatial overlap between the vehicle's 3D detection box and other traffic participants (such as dynamic vehicles and pedestrians) and static obstacles in the scene, accurately calculating the collision rate. Simultaneously, by combining dynamic risk parameters such as expected time of collision (TTC) and minimum safe distance, the system comprehensively evaluates the algorithm's robustness and active risk avoidance capabilities in extremely dangerous scenarios.
[0054] Finally, there are human factors-related indicators, mainly covering driving comfort and driving efficiency. The human factors discriminator starts from the real experience of human passengers, quantifying ride comfort by analyzing the smoothness of the vehicle's driving trajectory and the abruptness of acceleration and deceleration (such as the Jerk index for acceleration); at the same time, it quantifies driving efficiency by statistically analyzing the total time taken for the vehicle to reach the destination, average driving speed, and the success rate of passing through complex intersections.
[0055] The discriminator reads the vehicle's state, traffic participant states, map topology, traffic light states, and the actions of the algorithm under test from a unified timeline. At each time step, the collision discriminator calculates the intersection volume or separation distance between the vehicle's 3D bounding box and other target bounding boxes; a collision is defined as an intersection volume greater than zero or exceeding a threshold set based on sensor error. For moving targets, the expected collision time is also calculated based on relative position and relative velocity, and the minimum safe distance, near miss events, and collision events within the scene are counted. The same contact events in consecutive time steps are counted together to avoid duplicate counting.
[0056] The traffic rule discriminator projects the vehicle's location and trajectory onto a high-precision map or road topology, forming a rule-based state machine based on lane boundaries, center lines, driving direction, stop lines, drivable areas, and traffic light phases. Corresponding violation events are generated when the vehicle's detection frame or wheel track crosses a solid line by a set distance, crosses the stop line during a red light, travels in the opposite direction to the lane's permitted direction, enters a conflict zone without meeting yield conditions, or continuously deviates from the drivable area for more than a set time. Thresholds are read from the evaluation configuration, and tolerance bands can be set based on positioning errors.
[0057] The human factor discriminator calculates longitudinal and lateral acceleration based on the vehicle's speed sequence, and obtains the jerk by dividing the acceleration difference between adjacent moments by the time interval; the amplitude, duration, and frequency of exceeding the comfort threshold are used to form a comfort index. Driving efficiency is calculated based on the total time to reach the destination, effective driving distance, average speed, waiting time, and passage success rate; scenarios that end prematurely due to collisions or serious violations are counted as failures according to the configuration.
[0058] Each individual indicator is normalized according to predefined upper and lower bounds to ensure the result falls within the [0,1] interval; out-of-bounds values are truncated, and the original values are recorded. The overall safety result can be obtained by weighting collision risk, traffic regulation risk, comfort risk, and efficiency risk, with the weights fixed in the same evaluation task and saved with the report. The system simultaneously outputs time-step original indicators, scenario-level individual indicators, overall results, event timestamps, and evidence fragments, thereby supporting traceability, horizontal comparison, and threshold verification.
[0059] Through the comprehensive evaluation of the above multi-dimensional discriminators, this invention can generate a standardized and comprehensive driving safety evaluation report, which can not only accurately depict the driving behavior characteristics of different algorithms, but also provide a reliable quantitative basis for the horizontal comparison and iterative optimization of autonomous driving models.
[0060] This embodiment uses nuPlan real-world driving data as training and a realism reference. This data includes approximately 1,200 hours of driving records and 120 hours of driving video, covering four cities in the United States and Asia. During testing, scenarios were selected from outside the training set, with stratified sampling based on road type, traffic density, weather conditions, and safety risks. Each scenario used the same initial scenario, random seed, algorithm under test, and interaction duration.
[0061] The comparison schemes include: CARLA, a traditional simulation scheme based on graphics and physics engines, and VISTA 2.0, a scheme based on reconstruction of real data. Visual realism is measured by indicators such as image generation distance and video generation distance, while trajectory and condition control are measured by indicators such as absolute trajectory error, lane line F1 value, background target detection intersection-over-union ratio, time consistency L1 error, and structural similarity. Figure 4The quantitative results of the method of this invention were compared with those of CARLA and VISTA 2.0 in terms of visual realism and trajectory control accuracy. The experimental results show that the driving safety evaluation environment based on the world model has significant improvements in fidelity and controllability compared with the baseline model.
[0062] Example 2 To achieve the above embodiments, such as Figure 5 As shown, this embodiment also provides an autonomous driving safety evaluation device 10 based on a world model, the device 10 comprising: The world model generation module 100 is used to generate high-fidelity multi-view video frames and environmental states using a driving world model. The driving world model internalizes the distribution of real driving data into the model in a generative manner. The long-tail scene generation module 200 is used to generate safety-critical long-tail scenes ranging from simple to difficult based on a multimodal prompting lexicon and a reinforcement learning prompter, and to drive the traffic flow simulator to generate dynamic trajectories that conform to physical laws. The closed-loop simulation interaction module 300 is used to connect the autonomous driving model under test to the closed-loop simulation, receive the control actions output by it, and generate multi-view video frames and environmental states for the next moment by the driving world model based on the current environmental state and control actions, forming a closed-loop interaction of perception-decision-environmental feedback. The multi-dimensional evaluation module 400 is used to calculate collision risk, traffic rule violations, driving comfort and driving efficiency indicators in real time through a multi-dimensional discriminator, and output a comprehensive safety evaluation result.
[0063] An autonomous driving safety evaluation device based on a world model according to an embodiment of the present invention can effectively improve the efficiency and scene fidelity of autonomous driving safety evaluation, and solve the problems of low evaluation efficiency and large gap between simulation and reality in traditional methods.
[0064] To implement the methods of the above embodiments, the present invention also provides a computer device, such as... Figure 6 As shown, the computer device 600 includes a memory 601 and a processor 602; wherein, the processor 602 reads executable program code stored in the memory 601 to run a program corresponding to the executable program code, so as to implement the various steps of the method described above.
[0065] To implement the above embodiments, this application also proposes a non-transitory computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the method described in the foregoing embodiments.
[0066] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., refer to specific features, structures, materials, or characteristics described in connection with that embodiment or example, which are included in at least one embodiment or example of the present invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.
[0067] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this invention, "a plurality of" means at least two, such as two, three, etc., unless otherwise explicitly specified.
Claims
1. A safety evaluation method for autonomous driving based on a world model, characterized in that, include: A driving world model is constructed, which is used to generate high-fidelity multi-view video frames and environmental states, and the distribution of real driving data is internalized into the model in a generative manner. Based on a multimodal cue lexicon and a reinforcement learning cue, a targeted generation of safety-critical long-tail scenarios covering simple to difficult scenarios is achieved, and a traffic flow simulator is driven to generate dynamic trajectories that conform to physical laws. Perform closed-loop simulation, connect the autonomous driving model under test to the closed-loop simulation, receive the control actions output by it, and the driving world model generates multi-view video frames and environmental states for the next moment based on the current environmental state and control actions through autoregression, forming a closed-loop interaction of perception-decision-environmental feedback. The system uses a multi-dimensional discriminator to calculate collision risk, traffic rule violations, driving comfort, and driving efficiency in real time, and outputs a comprehensive safety assessment result.
2. The method as described in claim 1, characterized in that, Constructing the driving world model includes: The latent space video diffusion network is trained based on real road multi-view videos, vehicle motion and control records, traffic participant trajectories, high-precision maps, traffic signal status, and scene text descriptions. The training includes visual representation and video generation pre-training based on real videos, controllable generation training with multimodal conditions, causal closed-loop training with vehicle actions and historical states, and joint consistency training using cross-view geometric correspondence, instance identity preservation, and temporal continuity loss.
3. The method as described in claim 2, characterized in that, The pre-training for visual representation and video generation based on real videos includes: The continuous multi-view camera images are compressed into spatiotemporal latent variables by a video encoder. The spatiotemporal causal generation backbone predicts the latent variables of the next time step based on historical latent variables, current environmental state and vehicle control actions. The state prediction head synchronously outputs the structured states of vehicles, traffic participants, and road elements, and the video decoder restores the predicted latent variables into multi-view video frames.
4. The method as described in claim 1, characterized in that, The reinforcement learning cue is modeled as a Markov decision process, and its state space includes at least one or more of the following: road topology, motion state of the vehicle and background traffic participants, current multimodal cue features, and historical actions of the algorithm under test; its action space includes adjusting the motion parameters of background traffic participants and selecting or perturbing the multimodal cue conditions; the single-step reward is obtained by weighting and summing risk reward, authenticity reward, diversity reward, and physical constraint penalty with preset weights.
5. The method as described in claim 1, characterized in that, The closed-loop interaction is executed sequentially as follows: read the video and status of the previous moment, call the algorithm under test to obtain the control action, verify the action format and range, advance the traffic flow status, call the driving world model to generate the observation of the next moment, calculate the discrimination index and write it into the log; the interaction frequency is configured according to the planning cycle of the algorithm under test.
6. The method as described in claim 1, characterized in that, The multi-dimensional discriminator includes: The collision discriminator is used to calculate the spatial overlap between the vehicle detection box and other target detection boxes at each time step. When the overlap exceeds a preset threshold, it is recorded as a collision. The traffic rule discriminator is used to project the vehicle's location and trajectory onto the road topology and generate violation events based on lane boundaries, center lines, driving directions, stop lines, drivable areas, and traffic light phases. The human factor discriminator is used to calculate longitudinal and lateral acceleration and jerk based on the vehicle's speed sequence to form a comfort index.
7. The method as described in claim 6, characterized in that, The comprehensive safety assessment result is obtained by weighting collision risk, traffic regulation risk, comfort risk and efficiency risk. The weights are fixed in the same assessment task and saved with the report. The system also outputs time-step raw indicators, scenario-level individual indicators, comprehensive results, event timestamps and evidence fragments.
8. An autonomous driving safety evaluation device based on a world model, characterized in that, include: The world model generation module is used to generate high-fidelity multi-view video frames and environmental states using a driving world model. The driving world model internalizes the distribution of real driving data into the model in a generative manner. The long-tail scene generation module is used to generate safety-critical long-tail scenes ranging from simple to difficult based on a multimodal cue lexicon and a reinforcement learning cue, and to drive the traffic flow simulator to generate dynamic trajectories that conform to physical laws. The closed-loop simulation interaction module is used to connect the autonomous driving model under test to the closed-loop simulation, receive the control actions output by it, and generate multi-view video frames and environmental states for the next moment by the driving world model based on the current environmental state and control actions, forming a closed-loop interaction of perception-decision-environmental feedback. The multi-dimensional evaluation module is used to calculate collision risk, traffic rule violations, driving comfort and driving efficiency indicators in real time through a multi-dimensional discriminator, and output a comprehensive safety evaluation result.
9. A computer device, characterized in that, Including processor and memory; The processor reads executable program code stored in the memory to run a program corresponding to the executable program code, so as to implement the autonomous driving safety evaluation method based on the world model as described in any one of claims 1-7.
10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by the processor, the program implements an autonomous driving safety evaluation method based on a world model as described in any one of claims 1-7.
