An end-to-end automatic driving strategy self-evolution method based on a world model drive

Abstract

The application belongs to the technical field of automatic driving of automobiles, and discloses an end-to-end automatic driving strategy self-evolution method based on a world model driver, steps of which comprise: dividing a road topological structure into a plurality of road type scene pools; performing legal threat screening, and calling a visual language model to generate a threat trajectory; inputting historical observation, a trajectory of the automobile and the threat trajectory into a visual world model to generate multi-view visual observation at the next moment; an end-to-end automatic driving strategy model receives the observation and outputs a future trajectory of the automobile, builds a random strategy distribution and samples feedback to the visual world model to continue to promote the environment; a reward signal is calculated according to the trajectory of the automobile, and a model parameter is updated by using a proximal policy optimization algorithm; after each training stage ends, a weak point image is generated and fed back to the next round of training. The application realizes the training and test integrated closed loop self-evolution of the end-to-end automatic driving system, and improves the pertinence of long tail risk scene mining and the stability of reinforcement learning training.

Description

Technical Field

[0001] This invention relates to the field of autonomous driving technology, specifically to a self-evolutionary method for end-to-end autonomous driving strategies based on a world model. Background Technology

[0002] As autonomous driving technology evolves from traditional modular architectures to end-to-end architectures, the decision-making process of autonomous driving systems is gradually shifting from a phased, sequential processing of "perception, prediction, planning, and control" to deep neural networks directly outputting future driving trajectories or control actions from environmental observations. End-to-end autonomous driving systems can reduce manual rule design and error propagation between modules, demonstrating good driving decision-making capabilities in conventional road scenarios. However, in real-world open traffic environments, autonomous vehicles not only need to handle conventional scenarios such as straight driving, following other vehicles, and cruising, but also face a large number of low-frequency but high-risk long-tail scenarios, including unprotected steering at intersections, forced merging at ramps, low-speed interactions at roundabouts, lateral overtaking at complex intersections, and sudden intrusions by surrounding vehicles. These scenarios have low probability of occurrence, complex interaction relationships, and rapid risk evolution, making them difficult to fully cover through conventional data collection and offline supervised training. Therefore, how to construct an integrated training and testing framework that can automatically discover weaknesses in end-to-end autonomous driving systems in closed-loop environments and continuously update policy parameters based on test feedback has become a crucial technical challenge for improving the generalization capability of end-to-end autonomous driving systems in long-tail scenarios.

[0003] Currently, existing solutions for autonomous driving scenario mining and model evolution mainly include the following categories:

[0004] (1) End-to-end autonomous driving training scheme based on real data playback and offline supervised learning. This type of scheme is based on real road data collection, and takes multi-view images, LiDAR point clouds, vehicle status, high-precision maps, and trajectories of surrounding traffic participants as inputs, and uses human driving trajectories or dataset-labeled trajectories as supervision signals to train the end-to-end autonomous driving model to output the vehicle's future trajectory. This type of scheme has a clear training process and stable supervision signals, and can learn the conventional driving modes in real driving data well. However, its training method depends on the existing scenarios in the fixed dataset. The model can only learn from the real samples that have already appeared. It is difficult to actively generate new long-tail risk scenarios, and it is also difficult to automatically adjust the training focus based on the test failure results. It does not have a complete closed-loop self-evolution capability.

[0005] (2) Scene enhancement and autonomous driving testing schemes based on simulators or rule perturbations. These schemes modify weather, lighting, road adhesion conditions, traffic flow density, obstacle positions, surrounding vehicle speeds, or vehicle trajectories in a simulated environment to construct different test scenarios. Some methods generate interactive behaviors such as sudden braking, lane changing, insertion, and crossing based on preset rules to evaluate the performance of autonomous driving systems in dangerous scenarios. These schemes can generate a large number of test samples at a low cost and control the difficulty of the scenarios through parameterization. However, these methods typically rely on manually preset scripts or simple geometric perturbations, and the generated threatening behaviors are prone to deviating from the specific road topology. For example, generating ramp merging behavior in road environments unsuitable for forced merging, or generating lateral intersection conflicts in non-intersection scenarios, will lead to inconsistent scenario semantics. Furthermore, rule perturbations often only change local physical parameters, making it difficult to generate long-tail scenarios with high-order interactive logic and semantic rationality, and also difficult to adaptively generate scenarios that address the weaknesses of specific end-to-end models.

[0006] (3) Autonomous driving scene generation schemes based on world models or generative models. With the development of generative models and visual world models, some technical solutions utilize generative adversarial networks, diffusion models, video generation models, or autonomous driving world models to learn the spatiotemporal distribution of real driving scenes and generate future visual observations based on historical images, vehicle actions, high-precision maps, text prompts, or trajectory conditions. These schemes can generate high-fidelity multi-view images or video frames, providing a test environment closer to the real visual distribution for end-to-end autonomous driving systems. However, existing world model schemes mostly focus on scene generation quality or video prediction effects, and usually adopt an open-loop generation mode, that is, generating future segments all at once based on initial conditions, without using the current output trajectory of the vehicle in the end-to-end autonomous driving system as an environmental advancement condition. At this time, the world model plays more of a role in "generating test data" rather than a closed-loop environmental role in "advancing the environmental state according to policy actions", making it difficult to form a truly gradual interactive closed-loop training process.

[0007] In summary, the existing technical solutions have the following defects: (1) The offline supervised learning solution lacks a closed-loop feedback mechanism, and the test failure results cannot be automatically converted into training signals; (2) The rule perturbation solution lacks road topology constraints, the generated threat scenarios have poor semantic consistency, and cannot be adaptively generated for model weaknesses; (3) Most world model solutions are generated in an open loop, and there is a lack of step-by-step interaction relationship between them and the tested strategy, making it difficult to use as a reinforcement learning closed-loop environment to drive the end-to-end autonomous driving system for continuous strategy optimization. Summary of the Invention

[0008] To address the aforementioned technical problems, this invention provides a self-evolutionary method for end-to-end autonomous driving strategies based on a world model, comprising the following steps:

[0009] Step S1, Road topology-level scene partitioning

[0010] Obtain the original driving scenario data and divide the original driving scenario into several road type scenario pools based on the road topology;

[0011] The data includes multi-view image sequences, vehicle historical status, vehicle historical trajectory, historical and future trajectories of surrounding traffic participants, and high-precision map information.

[0012] Step S2, Legitimate Threat Screening and Threat Trajectory Generation

[0013] Within each road type scenario pool, a legitimate threat screening is performed to determine whether there are any risky traffic participants in the current scenario that meet the preset conditions. If not, the current scenario is marked as a normal scenario. If it exists, the current scenario is marked as a threat scenario, and the visual language model is called to generate an edited threat trajectory of the risky traffic participants based on the current road structure.

[0014] The current road structure includes road type, vehicle status, status of at-risk traffic participants, local high-precision map, basic future trajectory of at-risk traffic participants, and current vulnerability profile.

[0015] Step S3: Closed-loop advancement of the visual world model

[0016] Input the historical multi-view images, the vehicle's historical state, the state of surrounding vehicles, the layout of local high-precision maps or bird's-eye views, the vehicle's current planned trajectory output by the end-to-end autonomous driving strategy model, and the threat trajectory into the visual world model, and the visual world model will generate multi-view visual observations of the next moment or the short time domain future.

[0017] Step S4: Plug-in access to driving strategy and trajectory sampling

[0018] The end-to-end autonomous driving strategy model receives multi-view visual observations generated by the visual world model and outputs the vehicle's future trajectory; the vehicle's future trajectory is used as the mean of the strategy distribution to construct a random strategy distribution and sample to obtain the actual execution trajectory, and the actual execution trajectory is fed back to the visual world model to continue to advance the environmental state;

[0019] Step S5: Reward Calculation and Strategy Parameter Update

[0020] The reward signal is calculated based on the vehicle's trajectory, the trajectories of surrounding vehicles, and road structure information, and the training parameters of the end-to-end autonomous driving strategy model are updated using a near-end policy optimization algorithm.

[0021] Step S6, Weakness Profile Generation and Adaptive Sampling Feedback

[0022] After each road structure training phase, a weakness profile of the current end-to-end autonomous driving strategy model under the current road structure is generated based on the failure cases and evaluation indicators in the closed-loop test, and the weakness profile is fed back to the next round of threat scenario sampling and threat trajectory generation process.

[0023] Further, in step S1, the system divides the original driving scenario into intersection scenario, ramp or merging area scenario, roundabout scenario, straight road scenario, and curve scenario based on road topology information, and constructs corresponding road type scenario pools for each scenario. The road topology information includes several of the following: the vehicle's lane, lane connection relationships, intersection area, multi-lane merging area, roundabout area, road curvature, and the vehicle's historical driving direction.

[0024] Let the original driving scenario set be defined. for:

[0025]

[0026] in, Indicates the first One driving scenario, ;

[0027] For each driving scenario The system extracts its local high-precision map. Bicycle historical trajectory Surrounding vehicle status set And through road topology identification function Get road type label :

[0028]

[0029] Subsequently, the system stores the scenes into different road type scene pools based on the road type tags:

[0030]

[0031] in, Indicates the road type as Scene pool;

[0032] After the road type scenario pool is constructed, the system adopts a phased course training strategy for road structure; this strategy selects only one road structure scenario pool for closed-loop training and testing in each training phase.

[0033] Further, in step S2, for the current scenario, the system iterates through all traffic participants around the vehicle and determines whether they meet the conditions for being a risky traffic participant; the conditions include at least:

[0034] First, there is sufficient trajectory data for surrounding vehicles within a future timeframe;

[0035] Second, the spatial distance between the surrounding vehicles and the vehicle itself is within a preset reasonable range;

[0036] Third, the speed, acceleration, heading angle changes, and trajectory curvature of the surrounding vehicles satisfy the vehicle kinematic constraints;

[0037] Fourth, the surrounding vehicles are located in the drivable area or reasonable traffic area;

[0038] Fifth, there is no obvious semantic conflict between the movement direction of surrounding vehicles and the current road topology.

[0039] If there are no surrounding vehicles that meet the above conditions in the current scene, the scene is marked as a normal scene and enters the normal scene branch. In the normal scene branch, all traffic participants move according to the real trajectory in the dataset, which is mainly used to maintain the basic capabilities of the end-to-end autonomous driving model in normal driving scenarios.

[0040] If there are surrounding vehicles in the current scenario that meet at least one condition, the system marks that vehicle or the vehicle with the highest risk score as a risky traffic participant and allows the scenario to enter the threat scenario branch.

[0041] Furthermore, for samples entering the threat scenario branch, the system calls the visual language model to edit the threat trajectory; the input of the visual language model includes the current road type, vehicle status, risky traffic participant status, local high-precision map, basic future trajectory of risky traffic participants, and current vulnerability profile;

[0042] First, the above input is organized into structured prompts, which include fields for scene description, road structure, vehicle status, risky traffic participant status, basic trajectory, constraints, and vulnerability profile. The visual language model outputs a threat behavior description and an edited risky traffic participant threat trajectory based on these structured prompts. The threat behavior description includes at least one of the following: lateral jaywalking at an intersection, oncoming intrusion, unprotected left turn conflict, forced merging from a ramp, and slow insertion into a roundabout. The edited risky traffic participant threat trajectory... Represented as:

[0043]

[0044] in, Indicates the risk of traffic participants in the first The position of a future moment Indicates speed, Indicates the heading angle. Indicates the length of the trajectory within the predicted time range;

[0045] The following method is used to perform trajectory legality constraint checks on the threat trajectories generated by the visual language model:

[0046] The system determines whether the edited threat trajectory is located within the drivable area, whether it meets the constraints of speed, acceleration, jerk and curvature, whether it matches the road topology, and whether it forms a reasonable but not physically abnormal interactive risk. If the threat trajectory does not meet the constraints, the system performs a smooth correction, projects it to the drivable area, or re-invokes the visual language model to generate it.

[0047] Furthermore, in step S3, the visual world model receives historical multi-view images, the vehicle's historical state, the state of surrounding vehicles, the layout of local high-precision maps or bird's-eye view, the current planned trajectory of the end-to-end autonomous driving system, and the threat trajectory edited by the visual language model as conditional inputs, and generates multi-view visual observations for the next moment or the short time domain future.

[0048] Let the first The system state at any given time is Visual observation is The end-to-end autonomous driving strategy outputs the vehicle trajectory as follows: The visual language model generates the threat trajectory of the traffic participant at risk. The state progression process of the visual world model can then be represented as:

[0049]

[0050] in, This represents the visual observation at time t+1. Represents a model of the visual world. Represents a historical multi-view image sequence. Indicates the historical status of the vehicle. Indicates the status of surrounding traffic participants. This indicates the layout of a local high-definition map or bird's-eye view. Indicates the length of the historical time window.

[0051] Furthermore, in step S4, the end-to-end autonomous driving strategy model is accessed through a unified driving strategy plugin interface. The driving strategy plugin encapsulates different end-to-end autonomous driving models into a strategy module that "inputs environmental observation and outputs the future trajectory of the vehicle".

[0052] Let the parameters of the end-to-end autonomous driving strategy model be... Its input is the current state. The output is the average of the vehicle's future trajectory:

[0053]

[0054] in, This represents the mean trajectory output by the end-to-end autonomous driving strategy model. , This represents the longitudinal and lateral positions of the vehicle in the vehicle's coordinate system at the first future moment. , This represents the longitudinal and lateral positions of the vehicle in the vehicle coordinate system at the T-th future moment;

[0055] Construct a random policy distribution outside the model:

[0056]

[0057] in, Indicates the current environmental state Generate vehicle trajectory motion The random policy distribution, Indicates a Gaussian distribution. This represents the vehicle trajectory and actions sampled from the policy distribution. This represents the mean trajectory output by the end-to-end autonomous driving strategy model. This represents the exploration variance, which is either externally set or adaptively adjusted by the system; the above sampling yields the vehicle trajectory and motion. Feedback is sent to the visual world model to generate multi-view visual observations for the next moment;

[0058] During the rollout process, the system repeatedly executes the following steps over time: First, the visual world model generates the current or next-moment visual observation based on historical observations, vehicle actions, map information, and threat trajectories; second, the driving strategy plugin outputs the vehicle's future trajectory based on the current visual observation; then, the system constructs a random policy distribution based on this trajectory and samples the actual execution trajectory; subsequently, the sampled trajectory is fed back to the visual world model to continue advancing the environment; finally, the reward calculation module calculates the dense reward based on the current rollout result. Through continuous execution over multiple time steps, the system forms a progressively interactive visual rollout.

[0059] Furthermore, in step S5, dense rewards are calculated based on explicit geometric information during the closed-loop unfolding process; the reward items include at least collision risk, minimum distance between vehicles, collision time, whether the vehicle has left the drivable area, route advancement efficiency, speed rationality, trajectory smoothness, acceleration comfort, and curvature constraints.

[0060] No. Total reward of the moment Represented as:

[0061]

[0062] in, This indicates security rewards or penalties. This indicates rewards or penalties related to collision time. This indicates the drivable area constraint. This indicates the efficiency of route advancement. This indicates the reasonableness of the speed. Represents the trajectory smoothness term. This indicates the acceleration comfort item. Represents the curvature constraint term. , , , , , , , These are the weights of the corresponding items.

[0063] After completing a batch of scenarios or a fixed-length unfolding, the discounted reward and advantage function are calculated based on the reward at each time step, and the training parameters in the driving strategy plugin are updated using a proximal policy optimization algorithm; let the th... The advantage of time is estimated as The ratio of the probability of the new strategy to the probability of the old strategy for:

[0064]

[0065] in, and These represent the current updated driving strategy distribution and the old driving strategy distribution before the update, respectively.

[0066] The pruning target of the near-end strategy optimization Represented as:

[0067]

[0068] in, This represents the expectation of samples at each time step in a batch of unfolded scenarios. This represents the pruning function, used to restrict the ratio of the probabilities of the new and old strategies to a range. Inside; This is the cutting factor; This represents the advantage estimate at time t, used to measure the performance of the currently sampled vehicle trajectory relative to the average strategy.

[0069] In the parameter update phase, imitation learning replay loss and policy distribution regularization are introduced, with the overall training objective being... Represented as:

[0070]

[0071] in, This represents the imitation learning loss based on real driving data or the output of the original model. This represents the constraint on the difference between the distributions of the old and new strategies. and These are the weight coefficients for the corresponding items.

[0072] Furthermore, in step S6, a weakness profiling analysis is performed after each road structure training phase.

[0073] First, conduct closed-loop testing again in the current road type scenario pool, and count the collision rate, collision time violation rate, boundary crossing rate, route advancement efficiency, average speed, acceleration comfort, trajectory smoothness, and average reward.

[0074] Subsequently, failure cases are collected, including scenarios where collisions occur, scenarios where the collision time is below the threshold, scenarios where the vehicle leaves the drivable area, scenarios where the route advancement efficiency is too low, and scenarios where comfort is significantly reduced. For each failure case, the system records its road type, threat behavior type, risky traffic participant status, vehicle trajectory, risky traffic participant trajectory, failure time point, and corresponding reward breakdown results.

[0075] The vulnerability profiling analysis module generates a vulnerability profile of the current end-to-end autonomous driving strategy model based on the aforementioned failure cases. This vulnerability profile describes the road structure under which the current model is most likely to fail and what type of legitimate interaction threat it faces, thus constructing a risk matrix between road structure and threat patterns. :

[0076]

[0077] in, Indicates the road type. Indicates threat pattern, This indicates the collision rate under the combination of road structure and threat mode. Indicates the collision time violation rate. Indicates the out-of-bounds rate. This indicates a low percentage of rewarded samples. These are the weight coefficients for the corresponding items; the larger the matrix value, the more vulnerable the current end-to-end autonomous driving strategy model is to this type of road structure and threat pattern combination.

[0078] In the next round of training, the system dynamically adjusts the threat scenario sampling weights and the threat trajectory generation direction of the visual language model based on the vulnerability profile. For road structures and threat pattern combinations with high scores in the risk matrix, the system increases their threat scenario sampling probability and explicitly adds corresponding vulnerability information to the structured prompts, guiding the visual language model to generate more targeted and legitimate threat trajectories.

[0079] Regarding stage switching, the system determines whether to enter the next road structure training stage based on the closed-loop test indicators in the current road structure scenario pool. When the collision rate is lower than the preset threshold, the collision time violation rate is lower than the preset threshold, the average reward tends to converge, or the performance improvement is lower than the preset threshold in multiple consecutive training cycles, the system determines that the policy self-evolution training under the current road structure has been basically completed and switches to the next road structure scenario pool. After completing the training of multiple single road structure stages, the system re-mixes the already trained road type scenario pools and performs joint fine-tuning of multiple road structures.

[0080] Compared with the prior art, the present invention has the following beneficial effects:

[0081] This invention enables integrated closed-loop self-evolution of training and testing for end-to-end autonomous driving systems. Existing offline supervised learning solutions typically separate training and testing; the testing phase can only identify problems but cannot directly drive model parameter updates. This invention constructs a closed-loop process consisting of road scene segmentation, threat scene generation, closed-loop advancement of the visual world model, reward calculation, near-end policy optimization and update, and weakness profiling feedback. This allows failures such as collisions, boundary violations, collision timing violations, inefficiencies, and decreased comfort generated by the end-to-end autonomous driving model during closed-loop testing to be directly transformed into subsequent policy optimization signals. Therefore, the system no longer relies on repeated manual screening of failure cases and reorganization of training data, improving model capability repair efficiency and achieving the technical effect of "testing as training, failure as data, and weakness-driven re-evolution."

[0082] This invention improves the targeting and effectiveness of long-tail risk scenario mining. Traditional real-data replay methods can only utilize samples already present in the dataset, making it difficult to proactively construct new low-frequency, high-risk scenarios; while traditional rule-based perturbation methods can generate a large number of scenarios, they often lack specificity. This invention uses a weakness profiling analysis module to statistically analyze the failure performance of the current model under different road structures and threat modes, and dynamically adjusts the sampling weights and threat trajectory generation directions for the next round of threat scenarios based on the weakness profiling, gradually focusing the training process on the risk modes where the current model is weakest. This mechanism reduces the proportion of low-value samples and improves the utilization efficiency of long-tail scenario training data.

[0083] This invention improves the stability of closed-loop reinforcement learning training. Directly mixing all road structures and threat patterns for reinforcement learning training can easily lead to complex reward distributions, large policy gradient variance, and significant training oscillations. This invention employs a road topology-level phased training approach, allowing the end-to-end autonomous driving model to first learn relatively clear risk interaction patterns within a single road structure, such as training at intersections first, then training on ramps, roundabouts, and other scenarios. This approach reduces early training difficulty, enabling the model to gradually master interaction capabilities under different road structures, thereby improving the convergence stability and interpretability of closed-loop reinforcement learning training.

[0084] This invention features strong end-to-end model versatility and hot-swappability. Existing reinforcement learning training schemes typically require the design of specialized network structures or action spaces for specific autonomous driving models, making them difficult to transfer to other end-to-end models. This invention unifies different end-to-end autonomous driving systems into a driving policy plugin, requiring only that it can receive environmental observations and output the vehicle's future trajectory. The system uses the output trajectory as the mean of the policy distribution outside the model and constructs a random policy distribution for trajectory sampling and near-end policy optimization updates. Therefore, this invention can support UniAD, VAD, GenAD, and other trainable end-to-end autonomous driving models to access the same closed-loop self-evolutionary framework, reducing system adaptation costs.

[0085] This invention extends the visual world model from an open-loop video generator to a closed-loop training environment. Existing world models or generative models primarily focus on image realism, video prediction quality, or open-loop future segment generation, typically without updating the environment based on the real-time output trajectory of the end-to-end autonomous driving model. This invention uses the vehicle's planned trajectory output by the end-to-end autonomous driving model, the threat trajectory edited by the visual language model, historical multi-view images, the vehicle's state, the surrounding vehicle's state, and a high-precision map as conditional inputs to the visual world model. This allows the world model to progressively generate the next-moment visual observation based on the vehicle's policy actions and the threat behaviors of surrounding vehicles. Thus, the system forms a progressively interactive visual closed-loop rollout: "policy output trajectory—world model advancing environment—policy re-perception and decision," which is more suitable for closed-loop training and testing of end-to-end autonomous driving models. Attached Figure Description

[0086] Figure 1 This is a schematic diagram of the overall process of the self-evolution method for autonomous driving strategy of the present invention.

[0087] Figure 2 This is a schematic diagram of the road topology-level scene division and legitimate threat screening process of the present invention.

[0088] Figure 3 This is a schematic diagram of the threat trajectory editing based on the visual language model of the present invention.

[0089] Figure 4 This is a schematic diagram of the closed-loop advancement and strategy update process based on the visual world model of the present invention.

[0090] Figure 5 This is a schematic diagram illustrating the weakness profile generation and adaptive sampling feedback of the present invention. Detailed Implementation

[0091] The present invention will now be described in further detail with reference to the accompanying drawings and specific embodiments. The following embodiments are for illustrative purposes only and should not be considered as limiting the scope of the invention.

[0092] Example 1

[0093] This embodiment provides a world model-driven, end-to-end autonomous driving strategy self-evolution method, such as... Figure 1 As shown, it includes the following steps:

[0094] Step S1, Road topology-level scene partitioning

[0095] Obtain the original driving scenario data and divide the original driving scenario into several road type scenario pools based on the road topology;

[0096] The data includes multi-view image sequences, vehicle historical status, vehicle historical trajectory, historical and future trajectories of surrounding traffic participants, and high-precision map information.

[0097] Step S2, Legitimate Threat Screening and Threat Trajectory Generation

[0098] Within each road type scenario pool, a legitimate threat screening is performed to determine whether there are any risky traffic participants in the current scenario that meet the preset conditions. If not, the current scenario is marked as a normal scenario. If it exists, the current scenario is marked as a threat scenario, and the visual language model is called to generate an edited threat trajectory of the risky traffic participants based on the current road structure.

[0099] The current road structure includes road type, vehicle status, status of at-risk traffic participants, local high-precision map, basic future trajectory of at-risk traffic participants, and current vulnerability profile.

[0100] Step S3: Closed-loop advancement of the visual world model

[0101] Input the historical multi-view images, the vehicle's historical state, the state of surrounding vehicles, the layout of local high-precision maps or bird's-eye views, the vehicle's current planned trajectory output by the end-to-end autonomous driving strategy model, and the threat trajectory into the visual world model, and the visual world model will generate multi-view visual observations of the next moment or the short time domain future.

[0102] Step S4: Plug-in access to driving strategy and trajectory sampling

[0103] The end-to-end autonomous driving strategy model receives multi-view visual observations generated by the visual world model and outputs the vehicle's future trajectory; the vehicle's future trajectory is used as the mean of the strategy distribution to construct a random strategy distribution and sample to obtain the actual execution trajectory, and the actual execution trajectory is fed back to the visual world model to continue to advance the environmental state;

[0104] Step S5: Reward Calculation and Strategy Parameter Update

[0105] The reward signal is calculated based on the vehicle's trajectory, the trajectories of surrounding vehicles, and road structure information, and the training parameters of the end-to-end autonomous driving strategy model are updated using a near-end policy optimization algorithm.

[0106] Step S6, Weakness Profile Generation and Adaptive Sampling Feedback

[0107] After each road structure training phase, a weakness profile of the current end-to-end autonomous driving strategy model under the current road structure is generated based on the failure cases and evaluation indicators in the closed-loop test, and the weakness profile is fed back to the next round of threat scenario sampling and threat trajectory generation process.

[0108] Furthermore, such as Figure 2 As shown, unlike the method of directly classifying scenarios based on specific interactive behaviors such as emergency braking, lane cutting, and lane changing, in step S1 of this embodiment, the system classifies scenarios based on road topology. Specifically, the system divides the original driving scenario into intersection scenarios, ramp or merging area scenarios, roundabout scenarios, straight road scenarios, and curved road scenarios based on road topology information such as the vehicle's lane, lane connection relationships, intersection areas, multi-lane merging areas, roundabout areas, road curvature, and the vehicle's historical driving direction, and constructs corresponding road type scenario pools for each.

[0109] In one specific implementation, let the original driving scenario set be... for:

[0110]

[0111] in, Indicates the first One driving scenario, ;

[0112] For each driving scenario The system extracts its local high-precision map. Bicycle historical trajectory Surrounding vehicle status set And through road topology identification function Get road type label :

[0113]

[0114] in, The possible values ​​are road types such as intersection, ramp or merging area, roundabout, straight road, and curve;

[0115] Subsequently, the system stores the scenes into different road type scene pools based on the road type labels:

[0116]

[0117] in, Indicates the road type as The system can transform complex hybrid driving data into training resources with clear road structure semantics through the above processing, providing a data foundation for subsequent phased course training.

[0118] After the road type scenario pool is constructed, the system adopts a phased training strategy based on road structure. This strategy does not simultaneously mix all road structures for reinforcement learning training in the initial stage. Instead, it selects only one road structure scenario pool for closed-loop training and testing in each training stage. For example, in the first stage, the system only selects the intersection scenario pool for training, allowing the end-to-end autonomous driving system to focus on learning interaction risks such as lateral overtaking, unprotected left-turn conflicts, and oncoming vehicle intrusion. Once the collision rate, collision time violation rate, boundary crossing rate, and reward convergence in the intersection scenario meet preset conditions, the system switches to the ramp or merging area scenario pool to focus on training interaction risks such as forced merging, speed game theory, and lane-changing conflicts. This phased training approach, from a single road structure to multiple road structures, reduces the problems of overly complex reward distribution, high variance in policy gradients, and policy oscillations in the early stages of reinforcement learning training.

[0119] Furthermore, such as Figure 2 As shown, in step S2 of this embodiment, within each road type scenario pool, the system further performs legitimate threat screening; for the current scenario, the system iterates through all traffic participants around the vehicle and determines whether they meet the conditions for being a risky traffic participant; the conditions include at least:

[0120] First, there is sufficient trajectory data for surrounding vehicles within a future timeframe;

[0121] Second, the spatial distance between the surrounding vehicles and the vehicle itself is within a preset reasonable range;

[0122] Third, the speed, acceleration, heading angle changes, and trajectory curvature of the surrounding vehicles satisfy the vehicle kinematic constraints;

[0123] Fourth, the surrounding vehicles are located in the drivable area or reasonable traffic area;

[0124] Fifth, there is no obvious semantic conflict between the movement direction of surrounding vehicles and the current road topology.

[0125] If there are no surrounding vehicles that meet the above conditions in the current scene, the scene is marked as a normal scene and enters the normal scene branch. In the normal scene branch, all traffic participants move according to the real trajectory in the dataset, which is mainly used to maintain the basic capabilities of the end-to-end autonomous driving model in normal driving scenarios.

[0126] If there are surrounding vehicles in the current scenario that meet at least one condition, the system marks that vehicle or the vehicle with the highest risk score as a risky traffic participant and allows the scenario to enter the threat scenario branch. The risk score can be calculated based on indicators such as the minimum distance between the surrounding vehicles and the vehicle, relative speed, distance to potential intersections, estimated collision time, and degree of overlap of road conflict areas.

[0127] During the sampling process, the system samples within each road type scenario pool according to a preset ratio of normal scenarios to threat scenarios. In a preferred embodiment, the sampling ratio of normal scenarios to threat scenarios is set as follows:

[0128]

[0129] Here, "normal" represents ordinary scenarios, and "threat" represents threatening scenarios. Ordinary scenarios are used to maintain the model's adaptability to real-world driving distributions, while threatening scenarios are used to enhance the model's robustness in long-tail risk interactions. By setting this ratio, the system can avoid excessive exposure to adversarial scenarios during training, which could lead to a degradation of the model's normal driving capabilities, while ensuring that each training round includes a certain number of high-value risk samples.

[0130] Furthermore, such as Figure 3 As shown, for samples entering the threat scenario branch, the system calls the visual language model to edit the threat trajectory. The visual language model refers to a multimodal model that can simultaneously understand visual scenes, text commands, road semantics, and vehicle interaction relationships. The input of the visual language model includes the current road type, vehicle state, risky traffic participant state, local high-precision map, basic future trajectory of risky traffic participants, and current vulnerability profile. Among them, the vehicle state includes the vehicle's position, speed, acceleration, heading angle, and historical trajectory; the risky traffic participant state includes its current position, speed, heading angle, historical trajectory, and original future trajectory; the local high-precision map includes lane center lines, lane boundaries, drivable areas, intersection areas, merging areas, and traffic rule information; the vulnerability profile is used to describe under what road structures and threat modes the current end-to-end autonomous driving model is most likely to fail.

[0131] First, the above input is organized into structured prompt words, which include scene description field, road structure field, vehicle status field, risky traffic participant status field, basic trajectory field, constraint condition field, and weakness profile field.

[0132] The visual language model outputs a description of threatening behavior and an edited threat trajectory of the risky traffic participant based on the structured cue words; the description of threatening behavior includes at least one of the following: lateral crossing at an intersection, oncoming intrusion, unprotected left-turn conflict, forced merging from a ramp, and slow insertion into a roundabout; the edited threat trajectory of the risky traffic participant... Represented as:

[0133]

[0134] in, Indicates the risk of traffic participants in the first The position of a future moment Indicates speed, Indicates the heading angle. Indicates the length of the trajectory within the predicted time range;

[0135] Furthermore, to ensure that the threat trajectories generated by the visual language model have road semantic rationality and vehicle motion feasibility, a trajectory legality constraint check is performed on the threat trajectories generated by the visual language model, as follows:

[0136] The system determines whether the edited threat trajectory is located within a drivable area, whether it meets speed, acceleration, jerk, and curvature constraints, whether it matches the road topology, and whether it constitutes a reasonable but not physically abnormal interactive risk. If the threat trajectory does not meet the constraints, the system smoothly corrects it, projects it to a drivable area, or regenerates it using a visual language model. Through these methods, the present invention avoids generating unrealistic adversarial behavior in unreasonable road environments, ensuring that the threat trajectory always aligns with the current road structure and traffic semantics.

[0137] For example, during the training phase at intersections, the visual language model primarily generates threat trajectories such as lateral overtaking, oncoming vehicle intrusion, and intersections between straight-going and left-turning vehicles. During the training phase at ramps or merging areas, the visual language model primarily generates threat trajectories such as forced merging, short-distance lane changes, and speed-based maneuvering. During the training phase at roundabouts, the visual language model primarily generates threat trajectories such as yielding conflicts within the roundabout, low-speed insertion at the entrance, and vehicle interaction at the exit. Thus, the system implements a threat generation mechanism that "binds threat patterns to road structure."

[0138] Furthermore, such as Figure 4As shown, in step S3 of this embodiment, the present invention utilizes a visual world model to undertake the closed-loop environment advancement task. The visual world model receives historical multi-view images, the vehicle's historical state, the state of surrounding vehicles, the layout of local high-precision maps or bird's-eye views, the current planned trajectory of the end-to-end autonomous driving system, and the threat trajectory edited by the visual language model as conditional inputs, and generates multi-view visual observations for the next moment or the short-term future. In one specific implementation, the visual world model can adopt the OpenDWM model, or other world models that can generate future multi-view driving scenes based on historical visual observations, vehicle trajectory conditions, and map conditions.

[0139] Let the first The system state at any given time is Visual observation is The end-to-end autonomous driving strategy outputs the vehicle trajectory as follows: The visual language model generates the threat trajectory of the traffic participant at risk. The state progression process of the visual world model can then be represented as:

[0140]

[0141] in, Represents the visual observation at time t+1. Represents a model of the visual world. Represents a historical sequence of images from multiple perspectives. Indicates the historical status of the vehicle. Indicates the status of surrounding traffic participants. This indicates the layout of a local high-definition map or bird's-eye view. This indicates the length of the historical time window. Through the above-described condition generation method, the visual world model no longer predicts future scenes solely based on initial conditions in an open-loop manner, but rather progressively advances the environmental state based on the vehicle's trajectory and the threat trajectories of risky traffic participants currently output by the end-to-end autonomous driving strategy.

[0142] Furthermore, in step S4 of this embodiment, the end-to-end autonomous driving strategy model is connected to the system of the present invention through a unified driving strategy plugin interface. The driving strategy plugin encapsulates different end-to-end autonomous driving models into a strategy module that "inputs environmental observation and outputs the vehicle's future trajectory." This interface can connect to UniAD, VAD, GenAD, or other trainable end-to-end autonomous driving models. For any connected model, the system does not require it to change its original network structure, nor does it require the addition of a dedicated actor-critic network structure. Instead, it only requires that it be able to receive multi-view visual observations or bird's-eye view features generated by the visual world model and output the vehicle's future planned trajectory.

[0143] Let the parameters of the end-to-end autonomous driving strategy model be... Its input is the current state. The output is the average of the vehicle's future trajectory:

[0144]

[0145] in, This represents the mean trajectory output by the end-to-end autonomous driving strategy model. , This represents the longitudinal and lateral positions of the vehicle in the vehicle coordinate system at the first future moment. , This represents the longitudinal and lateral positions of the vehicle in the vehicle coordinate system at the T-th future moment;

[0146] To incorporate the original deterministic trajectory output into the reinforcement learning training process, this invention constructs a random policy distribution outside the model:

[0147]

[0148] in, Indicates the current environmental state Generate vehicle trajectory motion The random policy distribution, Indicates a Gaussian distribution. This represents the vehicle trajectory and actions sampled from the policy distribution. This represents the mean trajectory output by the end-to-end autonomous driving strategy model. This represents the exploration variance, which is either externally set or adaptively adjusted by the system; the above sampling yields the vehicle trajectory and motion. Feedback is fed into the visual world model to generate multi-view visual observations for the next moment. In this way, the present invention can incorporate the end-to-end model into the near-end policy optimization training framework without changing the original network structure of the end-to-end model, thereby achieving hot-swappable access to different end-to-end autonomous driving models.

[0149] During the rollout process, the system repeatedly executes the following steps over time: First, the visual world model generates the current or next-moment visual observation based on historical observations, vehicle actions, map information, and threat trajectories; second, the driving strategy plugin outputs the vehicle's future trajectory based on the current visual observation; then, the system constructs a random policy distribution based on this trajectory and samples the actual execution trajectory; subsequently, the sampled trajectory is fed back to the visual world model to continue advancing the environment; finally, the reward calculation module calculates the dense reward based on the current rollout result. Through the continuous execution of multiple time steps, the system forms a progressively interactive visual rollout.

[0150] Furthermore, such as Figure 4As shown, in step S5 of this embodiment, dense rewards are calculated based on explicit geometric information during the closed-loop unfolding process. These rewards do not rely on manually defined discrete scene labels, but are directly calculated based on the vehicle's trajectory, surrounding vehicle trajectories, drivable area, road centerline, vehicle bounding box, and traffic risk indicators. Reward items include at least collision risk, minimum distance between vehicles, collision time, whether the vehicle has left the drivable area, route advancement efficiency, speed rationality, trajectory smoothness, acceleration comfort, and curvature constraints.

[0151] In one specific implementation, the first Total reward of the moment Represented as:

[0152]

[0153] in, This indicates security rewards or penalties. This indicates rewards or penalties related to collision time. This indicates the drivable area constraint. This indicates the efficiency of route advancement. This indicates the reasonableness of the speed. Represents the trajectory smoothness term. This indicates the acceleration comfort item. Represents the curvature constraint term. , , , , , , , These are the weights of the corresponding items.

[0154] The safety bonus is calculated based on the minimum distance between the vehicle and the nearest surrounding vehicle. A larger negative bonus is applied when the vehicle collides with a surrounding vehicle; the closer the two vehicles are, the greater the risk penalty. The collision time bonus is calculated based on the relative distance and relative speed between the vehicle and surrounding vehicles; a penalty is applied when the expected collision time is below a preset threshold. The drivable area constraint penalizes the vehicle for straying beyond lane boundaries or leaving the drivable area. The route advancement efficiency encourages the vehicle to continue along the target route. The speed rationality constraint restricts the vehicle's speed to be close to the road speed limit or within a reasonable speed range. The trajectory smoothness and comfort constraints suppress rapid acceleration, deceleration, sharp turns, and excessive acceleration. The curvature constraint limits the curvature of the planned trajectory to ensure the trajectory meets vehicle kinematic feasibility requirements.

[0155] After completing a batch of scenarios or a fixed-length unfolding, the discounted reward and advantage function are calculated based on the reward at each time step, and the training parameters in the driving strategy plugin are updated using a proximal policy optimization algorithm; let the th... The advantage of time is estimated as The ratio of the probability of the new strategy to the probability of the old strategy for:

[0156]

[0157] in, and These represent the current updated driving strategy distribution and the old driving strategy distribution before the update, respectively.

[0158] The pruning target of the near-end strategy optimization Represented as:

[0159]

[0160] in, This represents the expectation of samples at each time step in a batch of unfolded scenarios. This represents the pruning function, used to restrict the ratio of the probabilities of the new and old strategies to a range. Inside; The pruning factor is used to limit the magnitude of a single policy update, preventing excessive changes in policy parameters. To ensure that the end-to-end autonomous driving model does not forget its original driving capabilities during reinforcement learning updates, the system introduces imitation learning replay loss and policy distribution regularization terms during the parameter update phase. The overall training objective is... Represented as:

[0161]

[0162] in, This represents the imitation learning loss based on real driving data or the output of the original model. This represents the constraint on the difference between the distributions of the old and new strategies. and These are the weight coefficients for the corresponding terms. Through this design, the system can improve robustness in long-tail scenarios while reducing the risk of degradation of original perception and planning abilities caused by reinforcement learning training.

[0163] To further improve training stability, this invention employs a phased parameter release strategy. In the minimum closed-loop verification phase, the system only updates planning-related parameters in the end-to-end autonomous driving strategy model, such as the planning head or trajectory regression branch parameters. After the closed-loop training stabilizes, the system gradually releases some intermediate feature fusion layers or prediction-planning related modules, entering a local fine-tuning phase. Once the system has achieved stable closed-loop training capabilities, it then supports full parameter fine-tuning of the model. In the full parameter fine-tuning phase, the system trains using a mixture of real driving data, ordinary closed-loop scenario data, and threat closed-loop scenario data, and combines imitation learning playback and policy regularization constraints to prevent catastrophic forgetting in the model.

[0164] Furthermore, such as Figure 5 As shown, in step S6 of this embodiment, a weakness profiling analysis is performed after each road structure training phase.

[0165] First, conduct closed-loop testing again in the current road type scenario pool, and count the collision rate, collision time violation rate, boundary crossing rate, route advancement efficiency, average speed, acceleration comfort, trajectory smoothness, and average reward.

[0166] Subsequently, failure cases are collected, including scenarios where collisions occur, scenarios where the collision time is below the threshold, scenarios where the vehicle leaves the drivable area, scenarios where the route advancement efficiency is too low, and scenarios where comfort is significantly reduced. For each failure case, the system records its road type, threat behavior type, risky traffic participant status, vehicle trajectory, risky traffic participant trajectory, failure time point, and corresponding reward breakdown results.

[0167] The vulnerability profiling analysis module generates a vulnerability profile of the current end-to-end autonomous driving strategy model based on the aforementioned failure cases. This vulnerability profile describes the road structure under which the current model is most likely to fail and what type of legitimate interaction threat it faces. For example, the system can obtain vulnerability descriptions such as "high risk of lateral lane cutting at intersections," "medium-high risk of forced merging at ramps," and "medium risk of low-speed insertion at roundabout entrances." To facilitate subsequent sampling and adjustment, a risk matrix is ​​constructed between road structure and threat patterns. :

[0168]

[0169] in, Indicates the road type. Indicates threat pattern, This indicates the collision rate under the combination of road structure and threat mode. Indicates the collision time violation rate. Indicates the out-of-bounds rate. This indicates a low percentage of rewarded samples. These are the weight coefficients for the corresponding items; the larger the matrix value, the more vulnerable the current end-to-end autonomous driving strategy model is to this type of road structure and threat pattern combination.

[0170] In the next round of training, the system dynamically adjusts the threat scene sampling weights and the direction of threat trajectory generation by the visual language model based on the weakness profile. For road structures and threat pattern combinations with high scores in the risk matrix, the system increases their threat scene sampling probability and explicitly adds corresponding weakness information to the structured prompts, guiding the visual language model to generate more targeted and legitimate threat trajectories. Through this mechanism, the training process can gradually focus on the risk patterns where the current model is weakest, thereby improving the training efficiency for long-tail scenarios.

[0171] Regarding stage switching, the system determines whether to enter the next road structure training stage based on the closed-loop test metrics in the current road structure scenario pool. When the collision rate is lower than a preset threshold, the collision time violation rate is lower than a preset threshold, the average reward tends to converge, or the performance improvement is lower than a preset threshold over multiple consecutive training cycles, the system determines that the policy self-evolution training under the current road structure is basically complete and switches to the next road structure scenario pool. For example, the system can sequentially complete the intersection stage, ramp or merging area stage, roundabout stage, curve stage, and straight road stage. After completing training for multiple single road structure stages, the system re-mixes the previously trained road type scenario pools and performs joint fine-tuning of multiple road structures to reduce the risk of the model forgetting previously learned road structures and further improve the model's generalization ability in complex mixed road environments.

[0172] Example 2

[0173] This embodiment, based on Embodiment 1, implements the method of the present invention using the nuScenes dataset and the UniAD end-to-end autonomous driving model.

[0174] The system first reads multi-view images, vehicle status, surrounding vehicle trajectories, and high-precision map information from the nuScenes trainval dataset, and then divides the scene into scene pools of road types such as intersections, ramps or merging areas, roundabouts, straight roads, and curves based on the local road topology.

[0175] During the training phase at intersections, the system samples training samples from the intersection scenario pool at a ratio of 8:1 for normal scenarios and threat scenarios. For normal scenarios, surrounding vehicles move along their actual trajectories; for threat scenarios, the system first filters out risky traffic participants that meet the constraints of trajectory length, spatial location, and motion state, and then calls the Vision-Language Model (VLM) to generate threat trajectories such as lateral overtaking, oncoming intrusion, or unprotected left-turn conflicts.

[0176] Subsequently, the system inputs historical multi-view images, the vehicle's historical state, the state of surrounding vehicles, a local high-precision map, the vehicle's currently planned trajectory output by UniAD, and the threat trajectory edited from the visual language model into OpenDWM. OpenDWM then generates multi-view visual observations for the next moment. UniAD receives these observations through the driving policy plugin interface and outputs the vehicle's future trajectory. The system uses this trajectory as the policy mean to construct a Gaussian policy distribution and samples the actual vehicle trajectory actions fed back to OpenDWM.

[0177] After completing closed-loop rollouts at multiple time steps, the system calculates rewards based on collision risk, collision time, boundary violations, efficiency, speed, smoothness, comfort, and curvature constraints, and updates UniAD's planning-related parameters using a near-end policy optimization algorithm. When the closed-loop test metrics in the intersection scenario meet the phase switching conditions, the system enters the ramp or merging area training phase and repeats the above process of scenario sampling, threat generation, world model advancement, policy output, reward calculation, and policy update.

[0178] Ultimately, through continuous training across multiple road structure stages and subsequent joint fine-tuning across multiple road structures, the system achieves continuous capability improvement for the end-to-end autonomous driving model in long-tailed interaction scenarios.

[0179] Example 3

[0180] This embodiment, based on Embodiment 1, offers alternative implementations for road topology recognition. It is not limited to rule-based map parsing methods. In practical applications, road topology recognition can employ deep learning-based road structure classification networks, graph neural network-based road topology recognition models, or hybrid recognition methods combining map vector elements and vehicle motion trajectories. Road type categories are not limited to intersections, ramps or merging areas, roundabouts, straight roads, and curves; they can be further expanded to include more granular road scenarios such as unprotected left-turn intersections, T-junctions, Y-junctions, construction zones, school zones, narrow road meeting areas, U-turn areas, and toll plaza areas.

Claims

1. A self-evolutionary method for end-to-end autonomous driving strategies based on a world model, characterized in that: Includes the following steps: Step S1: Obtain the original driving scenario data and divide the original driving scenario into several road type scenario pools according to the road topology. Step S2: Perform legitimate threat screening within each road type scenario pool to determine whether there are risky traffic participants in the current scenario that meet the preset conditions; if not, mark the current scenario as a normal scenario; if so, mark the current scenario as a threat scenario and call the visual language model to generate the edited risky traffic participant threat trajectory based on the current road structure. Step S3: Input the historical multi-view images, the vehicle's historical state, the surrounding vehicle state, the layout of the local high-precision map or bird's-eye view, the vehicle's current planned trajectory and threat trajectory output by the end-to-end autonomous driving strategy model into the visual world model, and the visual world model generates multi-view visual observations for the next moment or the short time domain future. Step S4: The end-to-end autonomous driving strategy model receives multi-view visual observations generated by the visual world model and outputs the vehicle's future trajectory; the vehicle's future trajectory is used as the mean of the strategy distribution to construct a random strategy distribution and sample to obtain the actual execution trajectory, and the actual execution trajectory is fed back to the visual world model to continue to advance the environmental state. Step S5: Calculate the reward signal based on the vehicle's trajectory, the trajectories of surrounding vehicles, and road structure information, and update the training parameters of the end-to-end autonomous driving strategy model using the near-end policy optimization algorithm; Step S6: After each road structure training phase, generate a weakness profile of the current end-to-end autonomous driving strategy model under the current road structure based on the failure cases and evaluation indicators in the closed-loop test, and feed the weakness profile back to the next round of threat scenario sampling and threat trajectory generation process. In step S6, a weakness profiling analysis is performed after each road structure training phase. First, conduct closed-loop testing again in the current road type scenario pool, and count the collision rate, collision time violation rate, boundary crossing rate, route advancement efficiency, average speed, acceleration comfort, trajectory smoothness, and average reward. Subsequently, failure cases are collected. For each failure case, the system records its road type, threat behavior type, risky traffic participant status, vehicle trajectory, risky traffic participant trajectory, failure time point, and corresponding reward breakdown results. The vulnerability profiling analysis module generates a vulnerability profile of the current end-to-end autonomous driving strategy model based on the aforementioned failure cases. This vulnerability profile describes the road structure under which the current model is most likely to fail and what type of legitimate interaction threat it faces, thus constructing a risk matrix between road structure and threat patterns. : in, Indicates the road type. Indicates threat pattern, This indicates the collision rate under the combination of road structure and threat mode. Indicates the collision time violation rate. Indicates the out-of-bounds rate. This indicates a low percentage of rewarded samples. These are the weight coefficients for the corresponding items; the larger the matrix value, the more vulnerable the current end-to-end autonomous driving strategy model is to this type of road structure and threat pattern combination. In the next round of training, the system dynamically adjusts the threat scenario sampling weights and the direction of threat trajectory generation by the visual language model based on the vulnerability profile. For road structures and threat pattern combinations with high scores in the risk matrix, the system increases their threat scenario sampling probability and explicitly adds corresponding vulnerability information to the structured prompts, guiding the visual language model to generate more targeted and legitimate threat trajectories.

2. The end-to-end autonomous driving strategy self-evolution method based on world model driving according to claim 1, characterized in that: In step S1, the system divides the original driving scenario into intersection scenario, ramp or merging area scenario, roundabout scenario, straight road scenario and curve scenario according to the road topology information, and constructs corresponding road type scenario pools for each. Let the original driving scenario set be For each driving scenario The system extracts its local high-precision map. Bicycle historical trajectory Surrounding vehicle status set And through road topology identification function Get road type label : Subsequently, the system stores the scenes into different road type scene pools based on the road type labels: in, Indicates the road type as Scene pool; After the road type scenario pool is constructed, the system adopts a phased course training strategy for road structure; this strategy selects only one road structure scenario pool for closed-loop training and testing in each training phase.

3. The end-to-end autonomous driving strategy self-evolution method based on world model driving according to claim 1, characterized in that: In step S2, for the current scenario, the system iterates through all traffic participants around the vehicle and determines whether they meet the conditions for being a risky traffic participant; the conditions include at least: First, there is sufficient trajectory data for surrounding vehicles within a future timeframe; Second, the spatial distance between the surrounding vehicles and the vehicle itself is within a preset reasonable range; Third, the speed, acceleration, heading angle changes, and trajectory curvature of the surrounding vehicles satisfy the vehicle kinematic constraints; Fourth, the surrounding vehicles are located in the drivable area or reasonable traffic area; Fifth, there is no obvious semantic conflict between the movement direction of surrounding vehicles and the current road topology; If there are no surrounding vehicles that meet the above conditions in the current scene, the scene is marked as a normal scene and enters the normal scene branch. In the normal scene branch, all traffic participants move according to the real trajectory in the dataset, which is mainly used to maintain the basic capabilities of the end-to-end autonomous driving model in normal driving scenarios. If there are surrounding vehicles in the current scenario that meet at least one condition, the system marks that vehicle or the vehicle with the highest risk score as a risky traffic participant and allows the scenario to enter the threat scenario branch.

4. A self-evolutionary method for end-to-end autonomous driving strategy based on a world model as described in claim 1 or 3, characterized in that: In step S2, for samples entering the threat scenario branch, the system calls the visual language model to edit the threat trajectory; the input of the visual language model includes the current road type, vehicle status, risky traffic participant status, local high-precision map, basic future trajectory of risky traffic participants, and current vulnerability profile; First, the above input is organized into structured cue words. The visual language model then outputs a description of the threat behavior and an edited threat trajectory of the at-risk traffic participant based on these structured cue words. Represented as: in, Indicates the risk of traffic participants in the first The position of a future moment Indicates speed, Indicates the heading angle. Indicates the length of the trajectory within the predicted time range; Perform trajectory legality constraint checks on threat trajectories generated by the visual language model.

5. The end-to-end autonomous driving strategy self-evolution method based on world model driving according to claim 1, characterized in that: In step S3, the visual world model receives historical multi-view images, the vehicle's historical state, the state of surrounding vehicles, the layout of local high-precision maps or bird's-eye view, the current planned trajectory of the end-to-end autonomous driving system, and the threat trajectory edited by the visual language model as conditional inputs, and generates multi-view visual observations for the next moment or the short time domain future. Let the first The system state at any given time is Visual observation is The end-to-end autonomous driving strategy outputs the vehicle trajectory as follows: The visual language model generates the threat trajectory of the traffic participant at risk. The state progression process of the visual world model can then be represented as: in, Represents the visual observation at time t+1. Represents a model of the visual world. Represents a historical sequence of images from multiple perspectives. Indicates the historical status of the vehicle. Indicates the status of surrounding traffic participants. This indicates the layout of a local high-definition map or bird's-eye view. Indicates the length of the historical time window.

6. The end-to-end autonomous driving strategy self-evolution method based on world model driving according to claim 1, characterized in that: In step S4, the end-to-end autonomous driving strategy model is accessed through a unified driving strategy plugin interface. The driving strategy plugin encapsulates different end-to-end autonomous driving models into a strategy module that "inputs environmental observation and outputs the future trajectory of the vehicle". Let the parameters of the end-to-end autonomous driving strategy model be... Its input is the current state. The output is the average of the vehicle's future trajectory: in, This represents the mean trajectory output by the end-to-end autonomous driving strategy model. , This represents the longitudinal and lateral positions of the vehicle in the vehicle coordinate system at the first future moment. , This represents the longitudinal and lateral positions of the vehicle in the vehicle coordinate system at the T-th future moment; Construct a random policy distribution outside the model: in, Indicates the current environmental state Generate vehicle trajectory motion The random policy distribution, Indicates a Gaussian distribution. This represents the vehicle trajectory and actions sampled from the policy distribution. This represents the mean trajectory output by the end-to-end autonomous driving strategy model. This represents the exploration variance, which is either externally set or adaptively adjusted by the system; the above sampling yields the vehicle trajectory and motion. Feedback is sent to the visual world model to generate multi-view visual observations for the next moment.

7. The end-to-end autonomous driving strategy self-evolution method based on world model driving according to claim 6, characterized in that: During the closed-loop deployment process, the system repeatedly executes the following process according to time steps: First, the visual world model generates the current or next moment's visual observation based on historical observations, vehicle actions, map information, and threat trajectories; second, the driving strategy plugin outputs the vehicle's future trajectory based on the current visual observation; then, the system constructs a random policy distribution based on this trajectory and samples it to obtain the actual execution trajectory. Subsequently, the sampling trajectory is fed back to the visual world model to continue advancing the environment; finally, the reward calculation module calculates the dense reward based on the current unfolding result; through the continuous execution of multiple time steps, the system forms a gradual interactive visual closed loop unfolding.

8. The end-to-end autonomous driving strategy self-evolution method based on world model driving according to claim 1, characterized in that: In step S5, dense rewards are calculated based on explicit geometric information during the closed-loop deployment process; the reward items include at least collision risk, minimum distance between vehicles, collision time, whether the vehicle has left the drivable area, route advancement efficiency, speed rationality, trajectory smoothness, acceleration comfort, and curvature constraints. After completing a batch of scenarios or a fixed-length unfolding, the discounted reward and advantage function are calculated based on the reward at each time step, and the training parameters in the driving strategy plugin are updated using a proximal policy optimization algorithm; let the th... The advantage of time is estimated as The ratio of the probability of the new strategy to the probability of the old strategy for: in, and These represent the current updated driving strategy distribution and the old driving strategy distribution before the update, respectively. The pruning target of the near-end strategy optimization Represented as: in, This represents the expectation of samples at each time step in a batch of unfolded scenarios. This represents the pruning function, used to restrict the ratio of the probabilities of the new and old strategies to a range. Inside; This is the cutting factor; This represents the advantage estimate at time t, used to measure the performance of the currently sampled vehicle trajectory relative to the average strategy. In the parameter update phase, imitation learning replay loss and policy distribution regularization are introduced, with the overall training objective being... Represented as: in, This represents the imitation learning loss based on real driving data or the output of the original model. This represents the constraint on the difference between the distributions of the old and new strategies. and These are the weight coefficients for the corresponding items.

9. The self-evolutionary method for end-to-end autonomous driving strategy based on world model driving according to claim 1, characterized in that: In terms of stage switching, the system determines whether to enter the next road structure training stage based on the closed-loop test indicators in the current road structure scenario pool. When the collision rate is lower than the preset threshold, the collision time violation rate is lower than the preset threshold, the average reward tends to converge, or the performance improvement is lower than the preset threshold in multiple consecutive training cycles, the system determines that the self-evolution training of the strategy under the current road structure has been basically completed and switches to the next road structure scenario pool. After completing training for multiple single road structure stages, the system re-mixes the trained road type scene pools and performs joint fine-tuning of multiple road structures.

Images