A self-evolutionary method for autonomous driving systems based on adversarial learning
By constructing a closed-loop system based on adversarial learning and utilizing the collaborative evolution of visual language action models and visual language models, the decision failure problem of autonomous driving systems in long-tail scenarios is solved, and the system's self-evolution and performance improvement are achieved.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- JILIN UNIVERSITY
- Filing Date
- 2026-05-08
- Publication Date
- 2026-07-17
AI Technical Summary
Existing autonomous driving systems suffer from decision failures or safety degradation when faced with unstructured long-tail scenarios that occur with extremely low probability, and existing technologies struggle to achieve closed-loop self-evolution of algorithms.
We employ an adversarial learning-based approach to construct a closed-loop system consisting of a cue-driven world model, a visual language action model (VLA), and a visual language model (VLM). We calculate the total loss value through a multi-dimensional evaluator and dynamically guide the system into different feedback loops to perform capability repair or boundary exploration. We then combine expert planning and human feedback reinforcement learning to fine-tune the model.
It has achieved continuous and automated performance improvement of autonomous driving system in long-tail scenarios, accurately repaired specific failure scenarios, broadened the boundary of model capabilities, and improved the driving performance of the system in complex scenarios.
Smart Images
Figure CN122174684B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of autonomous driving technology, and more specifically to a self-evolution method for autonomous driving systems based on adversarial learning. Background Technology
[0002] As autonomous driving technology evolves from a rule-driven modular architecture to a data-driven end-to-end architecture, the system's ability to generalize to complex scenarios has become a core indicator for evaluating its intelligence level. While end-to-end models perform excellently in handling routine conditions, they commonly suffer from decision failures or safety degradation when faced with extremely low-probability, unstructured "long-tail scenarios." Traditional methods of manually collecting and labeling long-tail data are not only extremely costly but also struggle to cover the infinite number of edge cases. Therefore, constructing a simulation training system capable of automatically identifying potential risk scenarios and driving algorithmic self-evolution has become an industry consensus for overcoming the bottlenecks of fully autonomous driving technology.
[0003] Currently, existing solutions for autonomous driving scenario mining and model evolution can be mainly categorized into the following three types:
[0004] 1. Scene enhancement methods based on heuristic rules or geometric transformations. These methods were a primary means of addressing data scarcity in the early stages. Their core idea is to generate new scenes by perturbing or recombining existing road test data. Technically, this typically involves using a physics engine to randomly change weather, lighting, or add traffic participants and obstacles to existing simulation scenes. For example, by altering the starting point or speed of vehicle trajectories, conflict situations can be artificially created.
[0005] 2. Static Simulation Scene Construction Scheme Based on Generative Models. With the development of deep generative models, learning statistical distributions from massive video data using generative adversarial networks or latent diffusion models has become the mainstream approach for generating high-fidelity scenes. These models learn feature associations from massive amounts of real-world road test data, and can receive prompts, high-precision maps, or vehicle action commands as conditional inputs to predict future visual scene frames. For example, through cross-modal interaction technology, the model can transform the semantics of "following another car in the rain" described in text into a multi-view video stream containing complex light and shadow reflections and texture details, thus providing a closed-loop perception testing platform for autonomous driving models.
[0006] 3. Task search methods based on adversarial testing and reinforcement learning. To proactively identify weaknesses in autonomous driving algorithms, researchers have begun to introduce adversarial learning concepts, modeling scene generation as an optimization problem that minimizes the performance of the autonomous driving algorithm. These methods typically utilize deep reinforcement learning (DRL) strategies to control background vehicles in the simulated environment, causing them to exhibit aggressive or abnormal driving behavior, thereby triggering collisions in the algorithm under test.
[0007] Although the above technical routes have made some progress in the field of scene generation and testing, they still have the following limitations in realizing the closed-loop self-evolution of autonomous driving algorithms: (1) The methods based on heuristic rules and geometric transformations rely on manual pre-set scripts or simple random perturbations. The generated scenes are often limited to the known physical parameter space, making it difficult to generate edge cases with high-order semantic features and complex logical interactions. This results in the system exhibiting low specificity when mining unknown long-tail scenes and making it difficult to accurately reach the deep performance boundaries of autonomous driving agents; (2) Existing generative world models mostly adopt an "open-loop" working mode. Their generation process is mainly driven by prompt words or map priors, without referring to the actual driving performance of the autonomous driving model in the current scene. This results in a lack of guidance in the allocation of training resources and limited evolution efficiency; (3) Existing adversarial testing schemes focus on finding aggressive behaviors that cause collisions with the vehicle. Their goal is usually to "find errors." They lack a closed-loop path that directly transforms the discovered failure scenes into model capability improvements, making it difficult to achieve true algorithm self-evolution. Summary of the Invention
[0008] To address the aforementioned technical problems, this invention provides a self-evolutionary method for autonomous driving systems based on adversarial learning, comprising the following steps:
[0009] Step 1: Generate the initial scene;
[0010] The prompt words are input into the world model to generate a simulation environment base that includes multi-view video streams, high-precision maps, and 3D collision box information;
[0011] Step 2: Decision-making and multi-dimensional assessment at the defense end;
[0012] The defense unit, as the core execution unit of the self-evolving system, integrates a Visual Language Action Model (VLA) and a multi-dimensional evaluator.
[0013] The visual language action model receives multi-view video streams from the world model and directly outputs the predicted trajectory of the vehicle. The predicted trajectory then enters the multi-dimensional evaluator, which simultaneously receives structured environmental information from the world model. By performing collision detection, boundary crossing detection, comfort detection, and efficiency detection in parallel, the multi-dimensional evaluator comprehensively calculates the total loss value reflecting the current driving performance.
[0014] Step 3: Dynamic path selection and execution;
[0015] The total loss value calculated by the multi-dimensional evaluator is compared with a preset threshold. Based on the comparison result, the evolution path is divided into two heterogeneous feedback loops:
[0016] If the total loss value L total Greater than the preset threshold L threshold When the visual language action model makes a dangerous decision that could lead to a collision or violation in the current scenario, and the current scenario triggers the capability defects of the autonomous driving agent, it enters the supervised fine-tuning path of loop A. In loop A, the expert planner intervenes and generates an expert trajectory that conforms to physical constraints and driving logic for the current difficult case, and stores it in the difficult case dataset. This high-quality expert data then serves as the ground truth to guide the visual language action model to perform supervised fine-tuning, thereby achieving accurate repair and capability enhancement for specific error scenarios.
[0017] If the total loss value L total Less than the preset threshold L threshold If the current scene is deemed insufficiently challenging for the visual language action model and unable to provide effective evolutionary incentives, then loop B is entered for capability boundary analysis. In loop B, an attack end containing a visual language model (VLM) is constructed. The visual language model (VLM) analyzer receives the current driving scene video, identifies potential weaknesses through deep semantic analysis, and fine-tunes it using a human feedback-based reinforcement learning (RLHF) mechanism to output a more challenging scene description. These enhanced challenging descriptions are fed back to the prompt word module at the starting end, regenerating prompt words to drive the world model to generate more complex and interactive edge cases.
[0018] Through this collaborative evolution of continuously exploring boundaries on the attack side and continuously absorbing expert experience on the defense side, the continuous and automated improvement of driving performance in highly interactive scenarios can be achieved.
[0019] Further, in step 1, the user or system can provide an initial scene description prompt, which is input into the world model. The world model is a simulation engine that can generate high-fidelity, multi-view visual streams and structured environmental information. Based on the input prompt, a simulation environment base containing multi-view video streams, high-precision map information, and 3D collision box information of each traffic participant is generated.
[0020] During the interactive simulation, the world model not only generates the environmental base at the initial moment, but also receives the vehicle's predicted trajectory output by the visual language action model in real time. Through autoregressive generation or physics engine rendering, it dynamically updates and outputs multi-view video frames and 3D environmental states for the next time step, thus forming a realistic physical interaction closed loop.
[0021] Furthermore, in step 2, the multi-dimensional evaluator executes four detection tasks in parallel, namely:
[0022] Collision detection, combined with 3D collision box information provided by the world model, calculates whether the vehicle's trajectory overlaps with any static or dynamic obstacles in space and time. If an overlap occurs, an extremely high collision loss value is generated. ;
[0023] Boundary crossing detection, combined with lane line information from a high-precision map, determines whether the vehicle's trajectory exceeds the drivable area; if it does, an extremely high boundary crossing penalty value is generated. ;
[0024] Comfort testing calculates the rate of change of acceleration along the vehicle's trajectory to quantify ride comfort; the more drastic the change, the greater the loss of comfort. The higher;
[0025] Efficiency testing calculates the difference between the vehicle's speed and the road speed limit; the larger the difference, the greater the efficiency loss. The higher.
[0026] The multi-dimensional evaluator calculates a total loss value by weighting and summing four sub-items: collision, boundary violation, comfort, and efficiency. This total loss value is used to evaluate the performance of the visual language action model driving mode. The formula for calculating the total loss value is as follows:
[0027]
[0028] in, This is the total loss value. , , and These represent the weighting coefficients for each of the following sub-categories: collision, boundary crossing, comfort, and efficiency. , , and These represent the loss values calculated for each of the following sub-items: collision, boundary violation, comfort, and efficiency.
[0029] Furthermore, the process of loop A described in step 3 is as follows:
[0030] (1) Expert planner intervention: Extract the environmental information of the first frame as the initial state, and enable an expert planner based on the Frenet coordinate system. The planner keeps the original trajectory of the obstacle unchanged and plans a safe and efficient trajectory for the vehicle alone.
[0031] (2) Trajectory sampling and filtering: In the Frenet coordinate system, the longitudinal displacement and lateral offset of the vehicle are sampled in multiple dimensions to generate a candidate trajectory set containing several candidate trajectories. , τ n Let n represent the nth candidate trajectory; then, perform a double validity check on each trajectory in the set: in the dynamic dimension, eliminate jerk-accelerated trajectories. or trajectory curvature Samples exceeding the physical limits of the vehicle; in the spatial dimension, the trajectory points in Frenet coordinates need to be restored to a Cartesian coordinate system. And by combining the vehicle's geometry to construct a spatiotemporal bounding box, it eliminates vehicles that are in the predicted time. The trajectory of the object overlaps spatially with the structured obstacles in the high-precision map.
[0032] (3) Multi-objective optimization: For each candidate trajectory that passes the screening, a multi-objective weighted cost function is introduced for quantitative evaluation, and the total cost is calculated. The calculation formula is as follows:
[0033]
[0034] in, For the sake of security, For the price of comfort, For the sake of efficiency, Cost of completion; ω s ω c ω e ω t These represent the weighting coefficients for safety costs, comfort costs, efficiency costs, and completion costs, respectively. Finally, the coefficients that minimize the total cost are selected. The smallest trajectory is taken as the optimal expert trajectory. , ;
[0035] (4) Monitor and fine-tune: Generate expert trajectory The corresponding scene The frame image sequence is extracted and combined with task prompts. This forms a training sample, which is stored in the hard example dataset. This hard example dataset is then used periodically for supervised fine-tuning of the visual language action model, minimizing the model's predicted trajectory. With expert trajectory The mean squared error loss is used to update the model parameters. The formula for calculating the mean squared error loss is as follows:
[0036] ;
[0037] Where L is the mean squared error loss. These are the expert trajectory and the predicted trajectory at time step t, respectively;
[0038] Through this process, the visual language action model learns how to make correct decisions in similar dangerous scenarios, achieving "self-evolution" of its capabilities.
[0039] Furthermore, the calculation method for each cost in the multi-objective weighted cost function is as follows:
[0040] (1) Security cost The aim is to quantify the collision risk between a vehicle and obstacles in the environment, based on the principle that "the farther away from the obstacle, the lower the cost." For at any time The Euclidean distance between the vehicle's 3D collision box and the boundary of the nearest dynamic obstacle is calculated using the following formula:
[0041]
[0042] in This is the sensitivity coefficient; when the distance... As the cost decreases, the cost increases exponentially.
[0043] (2) Cost of comfort Passenger comfort is quantified by jerk, and the cost is minimized by minimizing jerk. At the discrete time step, let... For the vehicle displacement, the jerk in the discrete time step The cost function is defined as the third derivative of displacement with respect to time:
[0044]
[0045] (3) Efficiency cost The goal is to minimize the cost of getting as close to the speed limit as possible, by calculating the instantaneous speed. Speed limits on roads This is achieved using the mean square error:
[0046]
[0047] (4) Cost of completion By calculating the lateral offset of the vehicle from the lane centerline in the Frenet coordinate system To achieve the goal of "the closer to the lane centerline, the lower the cost," the function is defined as:
[0048] .
[0049] Furthermore, the loop B process described in step 3 is as follows:
[0050] (1) Supervised fine-tuning: After supervised fine-tuning using a pre-built benchmark description dataset, the visual language model (VLM) is initially trained to transform from simple cue words to complex cue words, thereby establishing the initial policy model for subsequent reinforcement learning. The benchmark description dataset includes everything from simple basic scene cues to high-level descriptions containing complex environmental elements.
[0051] (2) Training the reward model: This reward model is based on the backbone network architecture of the large language model, and the terminal prediction layer is replaced with a linear mapping layer to output a scalar reward value. According to the initial strategy For specific guided inputs, multiple candidate scene descriptions are generated, and these outputs are then manually ranked and labeled according to preferences to form a human preference dataset. Based on this human preference dataset, a reward model is trained. The loss function is expressed as the negative log-likelihood function:
[0052]
[0053] in, For the loss function, The Sigmoid activation function is used, and this formula aims to optimize the model parameters. This makes it possible for the same input High-quality answer The reward value received is significantly higher than that for poor-quality answers. ; Expressing expectations;
[0054] (3) Proximal Policy Optimization (PPO) parameter update: The fine-tuned Visual Language Model (VLM) acts as the policy network (Actor), receiving the scene video information at the current moment as the environmental state input, and outputting cue words and actions aimed at increasing the difficulty of the scene accordingly; the previously trained reward model acts as the value network (Critic), evaluating the value of the cue words and actions generated by the policy network (Actor) and feeding back the corresponding scalar reward. Based on the reward signal from the reward model, the network parameters of the visual language model are iteratively updated using a proximal strategy optimization algorithm, ultimately resulting in an optimized visual language model capable of stably outputting prompts for high-difficulty scene descriptions.
[0055] Through the alternating execution of loop A and loop B, the visual language action model on the defensive end is specifically enhanced in its ability to handle known difficult scenarios, while the visual language model analyzer on the attacking end continuously explores and expands the capabilities of the defensive end. This closed-loop adversarial game process drives the continuous and automated evolution of the autonomous driving system's performance.
[0056] The beneficial effects of this invention are:
[0057] This invention proposes an autonomous driving self-evolution framework based on adversarial learning. By constructing a scene generation end consisting of a "cue word-driven world model", a defense end consisting of a "visual language action model (VLA)", and an attack end consisting of a "visual language model (VLM) analyzer", a closed-loop system is formed from scene generation, performance evaluation to vulnerability discovery and strategy repair.
[0058] This invention proposes a closed-loop self-evolutionary architecture comprised of a prompt-driven world model, a visual language action model (VLA), and a visual language model (VLM). The key feature is the calculation of the total loss value of the VLA-predicted trajectory using a multi-dimensional evaluator. Based on the comparison between the total loss value and a preset threshold, the system is dynamically guided into heterogeneous feedback loops: when the total loss value is higher than the preset threshold, it enters "Loop A" for capability repair; when the loss value is lower than the threshold, it enters "Loop B" for boundary exploration. This architecture breaks down the disconnect between "error detection" and "error repair" in existing technologies, achieving fully automated closed-loop improvement.
[0059] This invention designs an expert trajectory generation and supervised fine-tuning method for high-loss failure scenarios. A candidate trajectory set is constructed through multi-dimensional sampling in the Frenet coordinate system. This set is filtered by strict kinematic (e.g., jerk, curvature) and spatial geometric (e.g., 3D collision box overlap) constraints, and the optimal expert trajectory is selected by combining a multi-objective weighted cost function that considers safety, comfort, efficiency, and completion costs. This design uses physically feasible expert trajectories as ground truth signals to perform supervised fine-tuning of the Visual Language Action Model (VLA) by minimizing cross-entropy loss. This effectively solves the problem of decision failure in end-to-end models when facing long-tailed edge cases, achieving accurate repair and capability enhancement for specific defect scenarios.
[0060] This invention proposes an adversarial scene mining method based on Human Feedback Reinforcement Learning (RLHF). It trains a reward model by manually ranking and labeling candidate scene descriptions according to preferences, and iteratively updates the Visual Language Model (VLM) as the policy network using the Proximal Policy Optimization (PPO) algorithm. This method enables the VLM to stably output more challenging high-order semantic cues to drive the world model based on video input from a simple scene, significantly improving the targeting of mining unknown long-tail scenes. It effectively overcomes the shortcomings of traditional heuristic rules or open-loop generation models, such as lack of guidance in resource allocation and difficulty in reaching deep performance boundaries. Attached Figure Description
[0061] Figure 1 This is a flowchart illustrating a self-evolution method for an autonomous driving system based on adversarial learning, provided by the present invention.
[0062] Figure 2 This is a schematic diagram of the quantitative evaluation mechanism of the multi-dimensional evaluator provided in an embodiment of the present invention.
[0063] Figure 3 This is a flowchart illustrating loop A provided in an embodiment of the present invention.
[0064] Figure 4 This is a flowchart illustrating loop B provided in an embodiment of the present invention. Detailed Implementation
[0065] This embodiment provides a self-evolutionary method for autonomous driving systems based on adversarial learning. Through dynamic game between the attacker and defender, a closed-loop self-evolutionary system is constructed, encompassing the entire process from scene generation to decision optimization. The method includes the following steps:
[0066] Step 1: Generate the initial scene
[0067] The prompt words are input into the world model to generate a simulation environment base that includes multi-view video streams, high-precision maps, and 3D collision box information;
[0068] Specifically, as shown in the attached document Figure 1 As shown, the self-evolutionary process begins with a cue word; the user or system can provide an initial scene description cue word, such as "sunny day, city road, follow the car in front." This cue word is input into a world model, which is a simulation engine capable of generating high-fidelity, multi-view visual streams and structured environmental information. Based on the input cue word, a simulation environment base is generated, containing multi-view video streams, high-precision map information, and 3D collision box information of each traffic participant.
[0069] During the interactive simulation, the world model not only generates the environmental base at the initial moment, but also receives the vehicle's predicted trajectory output by the visual language action model in real time. Through autoregressive generation or physics engine rendering, it dynamically updates and outputs multi-view video frames and 3D environmental states for the next time step, thus forming a realistic physical interaction closed loop.
[0070] Step 2: Defense-side decision-making and multi-dimensional assessment
[0071] The defense unit, as the core execution unit of the self-evolving system, integrates a Visual Language Action Model (VLA) and a multi-dimensional evaluator.
[0072] The visual language action model receives multi-view video streams from the world model, processes them through its internal neural network, and directly outputs the predicted trajectory of the vehicle over a future period. The predicted trajectory then enters the multi-dimensional evaluator, which simultaneously receives structured environmental information from the world model. By performing collision detection, boundary crossing detection, comfort detection, and efficiency detection in parallel, the multi-dimensional evaluator comprehensively calculates the total loss value reflecting the current driving performance.
[0073] Specifically, as shown in the attached document Figure 2 As shown, the multi-dimensional evaluator executes four detection tasks in parallel:
[0074] Collision detection, combined with 3D collision box information provided by the world model, calculates whether the vehicle's trajectory overlaps with any static or dynamic obstacles in space and time. If an overlap occurs, an extremely high collision loss value is generated. ;
[0075] Boundary crossing detection, combined with lane line information from a high-precision map, determines whether the vehicle's trajectory exceeds the drivable area; if it does, an extremely high boundary crossing penalty value is generated. ;
[0076] Comfort testing calculates the rate of change of acceleration along the vehicle's trajectory to quantify ride comfort; the more drastic the change, the greater the loss of comfort. The higher;
[0077] Efficiency testing calculates the difference between the vehicle's speed and the road speed limit; the larger the difference, the greater the efficiency loss. The higher.
[0078] The multi-dimensional evaluator calculates a total loss value by weighting and summing four sub-items: collision, boundary violation, comfort, and efficiency. This total loss value is used to evaluate the performance of the visual language action model driving mode. The formula for calculating the total loss value is as follows:
[0079]
[0080] in, This is the total loss value. , , and These represent the weighting coefficients for each of the following sub-categories: collision, boundary crossing, comfort, and efficiency. , , and These represent the calculated loss values for collision, boundary violation, comfort, and efficiency, respectively. In practical applications, collision weights are set to ensure driving safety. Overweight The penalty weight is a relatively high order of magnitude to reflect its priority as a "hard constraint".
[0081] Step 3: Dynamic Path Selection and Execution
[0082] The total loss value calculated by the multi-dimensional evaluator is compared with a preset threshold. Based on the comparison result, the evolution path is divided into two heterogeneous feedback loops:
[0083] If the total loss value L total Greater than the preset threshold L threshold When the visual language action model makes a dangerous decision that could lead to a collision or violation in the current scenario, and the current scenario triggers the capability defects of the autonomous driving agent, it enters the supervised fine-tuning path of loop A. In loop A, the expert planner intervenes and generates an expert trajectory that conforms to physical constraints and driving logic for the current difficult case, and stores it in the difficult case dataset. This high-quality expert data then serves as the ground truth to guide the visual language action model to perform supervised fine-tuning, thereby achieving accurate repair and capability enhancement for specific error scenarios.
[0084] If the total loss value L total Less than the preset threshold L threshold If the current scene is deemed insufficiently challenging for the visual language action model and unable to provide effective evolutionary incentives, then loop B is entered for capability boundary analysis. In loop B, an attack end containing a visual language model (VLM) is constructed. The visual language model (VLM) analyzer receives the current driving scene video, identifies potential weaknesses through deep semantic analysis, and fine-tunes it using a human feedback-based reinforcement learning (RLHF) mechanism to output a more challenging scene description. These enhanced challenging descriptions are fed back to the prompt word module at the starting end, regenerating prompt words to drive the world model to generate more complex and interactive edge cases.
[0085] Through this collaborative evolution of continuously exploring boundaries on the attack end (visual language model) and continuously absorbing expert experience on the defense end (visual language action model), the continuous and automated improvement of driving performance in highly interactive scenarios can be achieved.
[0086] Specifically, as shown in the attached document Figure 3 As shown, the process of loop A (capability repair loop) is as follows:
[0087] (1) Expert planner intervention: Extract the environmental information of the first frame as the initial state, and enable an expert planner based on the Frenet coordinate system. The planner keeps the original trajectory of the obstacle unchanged and plans a safe and efficient trajectory for the vehicle alone.
[0088] (2) Trajectory sampling and filtering: In the Frenet coordinate system, the longitudinal displacement and lateral offset of the vehicle are sampled in multiple dimensions to generate a candidate trajectory set containing several candidate trajectories. Then, perform a double validity check on each trajectory in the set: in the dynamic dimension, remove jerk (jump). or trajectory curvature Samples exceeding the physical limits of the vehicle; in the spatial dimension, the trajectory points in Frenet coordinates need to be restored to a Cartesian coordinate system. And by combining the vehicle's geometry to construct a spatiotemporal bounding box, it eliminates vehicles that are in the predicted time. The trajectory of the object overlaps spatially with the structured obstacles in the high-precision map.
[0089] (3) Multi-objective optimization: For each candidate trajectory that passes the screening, a multi-objective weighted cost function is introduced for quantitative evaluation, and the total cost is calculated. The calculation formula is as follows:
[0090]
[0091] in, For the sake of security, For the price of comfort, For the sake of efficiency, Cost of completion; ω s ω c ω e ω t These represent the weighting coefficients for safety costs, comfort costs, efficiency costs, and completion costs, respectively. Finally, the coefficients that minimize the total cost are selected. The smallest trajectory is taken as the optimal expert trajectory. , .
[0092] (4) Monitor and fine-tune: Generate expert trajectory The corresponding scene The frame image sequence is extracted and combined with task prompts. (e.g., "Plan future trajectory"), forming a training sample, and storing it in the hard example dataset; periodically use this hard example dataset to perform supervised fine-tuning of the visual language action model, minimizing the model's predicted trajectory. With expert trajectory The mean squared error loss is used to update the model parameters. The formula for calculating the mean squared error loss is as follows:
[0093] ;
[0094] Where L is the mean squared error loss. These are the expert trajectory and the predicted trajectory at time step t, respectively;
[0095] Through this process, the visual language action model learns how to make correct decisions in similar dangerous scenarios, achieving "self-evolution" of its capabilities.
[0096] Furthermore, the calculation method for each cost in the multi-objective weighted cost function is as follows:
[0097] (1) Security cost The aim is to quantify the collision risk between a vehicle and obstacles in the environment, based on the principle that "the farther away from the obstacle, the lower the cost." For at any time The Euclidean distance between the vehicle's 3D collision box and the boundary of the nearest dynamic obstacle is calculated using the following formula:
[0098]
[0099] in Let exp represent the sensitivity coefficient as an exponential function with the natural constant e as the base; when the distance... As the cost decreases, the cost increases exponentially.
[0100] (2) Cost of comfort Passenger comfort is quantified by jerk, and the cost is minimized by minimizing jerk. At the discrete time step, let... For the vehicle displacement, the jerk in the discrete time step The cost function is defined as the third derivative of displacement with respect to time:
[0101]
[0102] (3) Efficiency cost The goal is to minimize the cost of getting as close to the speed limit as possible, by calculating the instantaneous speed. Speed limits on roads This is achieved using the mean square error:
[0103]
[0104] (4) Cost of completion By calculating the lateral offset of the vehicle from the lane centerline in the Frenet coordinate system To achieve the goal of "the closer to the lane centerline, the lower the cost," the function is defined as:
[0105]
[0106] Specifically, as shown in the attached document Figure 4 As shown, the process of loop B (boundary exploration loop) is as follows:
[0107] (1) Supervised fine-tuning: After supervised fine-tuning using a pre-built benchmark description dataset, the visual language model (VLM) is initially trained to transform from simple cue words to complex cue words, thereby establishing the initial policy model for subsequent reinforcement learning. The benchmark description dataset includes everything from simple basic scene cues (such as “Sunny, follow the car”) to higher-order expressions that include complex environmental elements (such as “Rainy, follow the car”).
[0108] (2) Training the reward model: This reward model is based on the backbone network architecture of the large language model, and the terminal prediction layer is replaced with a linear mapping layer to output a scalar reward value. According to the initial strategy For specific guiding inputs (such as "The current scenario is too simple... provide a more challenging description"), samples are taken to generate multiple candidate scenario descriptions. These outputs are then manually ranked and labeled according to qualitative preferences (e.g., labeling outputs as high-quality answers). Better than a poor answer This creates a human preference dataset; based on this dataset, a reward model is trained. The loss function is expressed as the negative log-likelihood function:
[0109]
[0110] in, For the loss function, The Sigmoid activation function is used, and this formula aims to optimize the model parameters. This makes it possible for the same input High-quality answer The reward value received is significantly higher than that for poor-quality answers. ; Expressing expectations;
[0111] (3) Proximal Policy Optimization (PPO) parameter update: The fine-tuned Visual Language Model (VLM) acts as the policy network (Actor), receiving the scene video information at the current moment as the environmental state input, and outputting cue words and actions aimed at increasing the difficulty of the scene accordingly; the previously trained reward model acts as the value network (Critic), evaluating the value of the cue words and actions generated by the policy network (Actor) and feeding back the corresponding scalar reward. Based on the reward signal from the reward model, the network parameters of the visual language model are iteratively updated using a proximal strategy optimization algorithm, ultimately resulting in an optimized visual language model capable of stably outputting prompts for high-difficulty scene descriptions.
[0112] Through the alternating execution of loop A and loop B, the visual language action model on the defensive end is specifically enhanced in its ability to handle known difficult scenarios, while the visual language model analyzer on the attacking end continuously explores and expands the capabilities of the defensive end. This closed-loop adversarial game process drives the continuous and automated evolution of the autonomous driving system's performance.
[0113] The above description is merely a preferred embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.
Claims
1. A self-evolutionary method for autonomous driving systems based on adversarial learning, characterized in that: Includes the following steps: Step 1: Input the prompt words into the world model to generate a simulation environment base containing multi-view video streams, high-precision maps, and 3D collision box information; Step 2: The visual language action model receives multi-view video streams from the world model and directly outputs the predicted trajectory of the vehicle; the predicted trajectory enters the multi-dimensional evaluator, which simultaneously receives structured environmental information fed back by the world model, and calculates the total loss value reflecting the current driving performance by performing collision detection, boundary crossing detection, comfort detection and efficiency detection in parallel. Step 3: Compare the total loss value calculated by the multi-dimensional evaluator with the preset threshold. Based on the comparison results, divide the evolution path into: If the total loss value exceeds the preset threshold, the supervised fine-tuning path of loop A is entered: the expert planner intervenes and generates an expert trajectory that conforms to physical constraints and driving logic for the current difficult case, and stores it in the difficult case dataset; then it is used as the ground truth to guide the visual language action model to perform supervised fine-tuning, so as to achieve accurate repair and capability enhancement for specific error scenarios; If the total loss value is less than the preset threshold, enter loop B to perform capability boundary analysis: construct an attack end containing a visual language model, the visual language model analyzer receives the current driving scene video, identifies potential weaknesses through deep semantic analysis, and uses a reinforcement learning mechanism based on human feedback to fine-tune it, outputting a more challenging scene description to the starting end, and regenerating prompt words to drive the world model to generate more complex and interactive edge cases. Through continuous exploration of boundaries and the synergistic evolution of absorbing expert experience, we can achieve continuous and automated improvement in driving performance in highly interactive scenarios.
2. The self-evolutionary method for an autonomous driving system based on adversarial learning according to claim 1, characterized in that: In step 1, the user or system provides an initial scene description prompt, which is input into the world model. The world model is a simulation engine that can generate high-fidelity, multi-view visual streams and structured environmental information. Based on the input prompt, a simulation environment base containing multi-view video streams, high-precision map information, and 3D collision box information of each traffic participant is generated.
3. The self-evolutionary method for an autonomous driving system based on adversarial learning according to claim 2, characterized in that: During the interactive simulation, the world model not only generates the environmental base at the initial moment, but also receives the vehicle's predicted trajectory output by the visual language action model in real time. Through autoregressive generation or physics engine rendering, it dynamically updates and outputs multi-view video frames and 3D environmental states for the next time step, thus forming a realistic physical interaction closed loop.
4. The self-evolution method for an autonomous driving system based on adversarial learning according to claim 1, characterized in that: In step 2, the multi-dimensional evaluator executes four detection tasks in parallel: Collision detection, combined with 3D collision box information provided by the world model, calculates whether the vehicle's trajectory overlaps with any static or dynamic obstacles in time and space; Boundary crossing detection, combined with lane line information in high-precision maps, determines whether the vehicle's trajectory exceeds the drivable area; Comfort testing calculates the rate of change of acceleration along the vehicle's trajectory to quantify ride comfort; Efficiency detection calculates the difference between the vehicle's speed and the road speed limit.
5. The self-evolutionary method for an autonomous driving system based on adversarial learning according to claim 1, characterized in that: In step 2, the multi-dimensional evaluator calculates the total loss value by weighting and summing the four sub-items: collision, boundary violation, comfort, and efficiency. This total loss value is used to evaluate the performance of the visual language action model driving mode. The formula for calculating the total loss value is as follows: in, This is the total loss value. , , and These represent the weighting coefficients for each of the following sub-categories: collision, boundary crossing, comfort, and efficiency. , , and These represent the loss values calculated for each of the following sub-items: collision, boundary crossing, comfort, and efficiency.
6. The self-evolutionary method for an autonomous driving system based on adversarial learning according to claim 1, characterized in that: The process of loop A in step 3 is as follows: (1) Expert planner intervention: Extract the environmental information of the first frame as the initial state, and enable an expert planner based on the Frenet coordinate system. The planner keeps the original trajectory of the obstacle unchanged and plans a safe and efficient trajectory for the vehicle alone. (2) Trajectory sampling and filtering: In the Frenet coordinate system, the longitudinal displacement and lateral offset of the vehicle are sampled in multiple dimensions to generate a candidate trajectory set containing several candidate trajectories. , τ n Let n be the nth candidate trajectory; then, perform a double validity check on each trajectory in the set: in the dynamic dimension, eliminate jerk-accelerated trajectories. or trajectory curvature Samples exceeding the physical limits of the vehicle; in the spatial dimension, the trajectory points in Frenet coordinates need to be restored to a Cartesian coordinate system. And by combining the vehicle's geometry to construct a spatiotemporal bounding box, it eliminates vehicles that are in the predicted time. The trajectory of the object overlaps spatially with the structured obstacles in the high-precision map; (3) Multi-objective optimization: For each candidate trajectory that passes the screening, a multi-objective weighted cost function is introduced for quantitative evaluation, and the total cost is calculated. The calculation formula is as follows: in, For the sake of security, For the price of comfort, For the sake of efficiency, Cost of completion; ω s ω c ω e ω t These represent weighting coefficients for safety costs, comfort costs, efficiency costs, and completion costs, respectively; ultimately, the coefficients that minimize the total cost are selected. The smallest trajectory is taken as the optimal expert trajectory. , ; (4) Monitor and fine-tune: Generate expert trajectory The corresponding scene The frame image sequence is extracted and combined with task prompts. This forms a training sample, which is stored in the hard example dataset. This hard example dataset is then used periodically for supervised fine-tuning of the visual language action model, minimizing the model's predicted trajectory. With expert trajectory The mean squared error loss is used to update the model parameters. The formula for calculating the mean squared error loss is as follows: ； Where L is the mean squared error loss. These are the expert trajectory and the predicted trajectory at time step t, respectively; Through this process, the visual language action model learns how to make correct decisions in similar dangerous scenarios, achieving "self-evolution" of its capabilities.
7. The self-evolutionary method for an autonomous driving system based on adversarial learning according to claim 6, characterized in that: The calculation method for each cost in the multi-objective weighted cost function is as follows: (1) Security cost The aim is to quantify the collision risk between a vehicle and obstacles in the environment, based on the principle that "the farther away from the obstacle, the lower the cost." For at any time The Euclidean distance between the vehicle's 3D collision box and the boundary of the nearest dynamic obstacle is calculated using the following formula: in This is the sensitivity coefficient; when the distance... As the cost decreases, the cost increases exponentially. (2) Cost of comfort Passenger comfort is quantified by jerk, and the cost is minimized by minimizing jerk. At the discrete time step, let... For the vehicle displacement, the jerk in the discrete time step The cost function is defined as the third derivative of displacement with respect to time: (3) Efficiency cost The goal is to minimize the cost of getting as close to the speed limit as possible, by calculating the instantaneous speed. Speed limits on roads This is achieved using the mean square error: (4) Cost of completion By calculating the lateral offset of the vehicle from the lane centerline in the Frenet coordinate system To achieve the goal of "the closer to the lane centerline, the lower the cost," the function is defined as: 。 8. The self-evolutionary method for an autonomous driving system based on adversarial learning according to claim 1, characterized in that: The process of loop B described in step 3 is as follows: (1) Supervised fine-tuning: After supervised fine-tuning using a pre-built benchmark description dataset, the visual language model is initially trained to transform from simple cue words to complex cue words, thereby establishing the initial policy model for subsequent reinforcement learning. The benchmark description dataset includes everything from simple basic scene cues to high-level descriptions containing complex environmental elements. (2) Training the reward model: This reward model is based on the backbone network architecture of the large language model, and the terminal prediction layer is replaced with a linear mapping layer to output a scalar reward value. According to the initial strategy For specific guided inputs, multiple candidate scene descriptions are generated, and these outputs are then manually ranked and labeled according to preferences to form a human preference dataset. Based on this human preference dataset, a reward model is trained. The loss function is expressed as the negative log-likelihood function: in, For the loss function, The Sigmoid activation function is used, and this formula aims to optimize the model parameters. This makes it possible for the same input High-quality answer The reward value received was significantly higher than that for poor-quality answers. ; Expressing expectations; (3) Near-end policy optimization parameter update: The fine-tuned visual language model acts as the policy network, receives the scene video information at the current moment as the environmental state input, and outputs prompt words and actions aimed at increasing the difficulty of the scene accordingly; the previously trained reward model acts as the value network, evaluates the value of the prompt words and actions generated by the policy network and feeds back the corresponding scalar reward. Based on the reward signal from the reward model, the network parameters of the visual language model are iteratively updated using a proximal strategy optimization algorithm, ultimately resulting in an optimized visual language model capable of stably outputting prompts for high-difficulty scene descriptions.