Image-based and large model-based automatic driving adversarial test scene parameter generation method

Abstract

The application discloses an automatic driving confrontation test scene parameter generation method based on an image and a large model, and belongs to the technical field of artificial intelligence and automatic driving. The method comprises the following steps: constructing a double-layer dynamic capability image matrix of a tested host vehicle, which comprises a priori layer and experience layer; inputting the image matrix into a large language model to infer and output a scene strategy text; inputting the strategy text and the image matrix into a scene generation network, and converting the strategy text and the image matrix into a physical scene parameter vector by adopting a fast time scale parameter optimization mechanism; calculating a direct preference optimization loss and a cross-modal graph-text consistency loss, and jointly updating low-rank fine-tuning adapter parameters of the large language model in an asynchronous mode; wherein an error backdoor gating mechanism is introduced when the graph-text consistency loss is calculated, and gradient conduction is adaptively controlled according to an execution deviation rate. The application solves the problems of low cold start efficiency of confrontation training, semantic illusion of a large language model and responsibility allocation disorder, and improves the efficiency, accuracy and engineering executability of test scene generation.

Description

Technical Field

[0001] This invention belongs to the field of artificial intelligence and autonomous driving technology, and specifically relates to a method for generating parameters for autonomous driving adversarial test scenarios based on profiles and large models. Background Technology

[0002] In recent years, end-to-end autonomous driving models have demonstrated enormous application potential due to their concise architecture that directly translates sensor data into control commands. However, when these highly data-driven models are generalized to real-world open roads, they are highly susceptible to fatal failures in long-tail scenarios such as rare severe weather, complex road topologies, and unusual traffic participant behavior. Because collecting this long-tail accident data in the real world is costly and extremely dangerous, the industry has widely shifted towards generating adversarial test scenarios in closed-loop physics simulation engines (such as CARLA) to accelerate the exposure of the safety boundaries of autonomous driving models.

[0003] However, in the high-dimensional continuous physical world parameter space, how to efficiently and accurately generate test scenarios that "can both directly target the weaknesses of the main vehicle and conform to objective physical laws" is a core technical problem that urgently needs to be solved in the current field of autonomous driving testing.

[0004] Currently, similar implementation solutions mainly fall into the following two categories:

[0005] The first category is parameterization generation methods based on heuristic search and random sampling. This type of approach is the traditional paradigm widely used in industry. The system usually uses genetic algorithms or basic reinforcement learning algorithms to perform large-scale random mutations and sampling within a set parameter space (such as the initial coordinates, speed, weather parameters, etc. of traffic participants) in an attempt to find the parameter combination that can lead to a collision between the main vehicles.

[0006] The second category is the direct generation method of open-loop cue words based on large language models. With the rapid development of large model technology, some studies have begun to explore the use of large language models to directly generate text descriptions of traffic scenarios.

[0007] However, the above-mentioned existing technical solutions have the following obvious drawbacks:

[0008] First, the lack of prior knowledge about the tested vehicle makes the "cold start" phase of adversarial training inefficient and difficult to accurately pinpoint boundaries. Traditional random sampling or purely parametric scene generation methods treat the autonomous vehicle as a complete black box. During the cold start phase of adversarial training, the generator network lacks effective attack experience. Due to the extremely large parameter space of high-dimensional continuous physical scenes, blind random sampling and searching require extremely high time and computing power costs, and it is difficult to accurately and stably locate the core capability boundaries of the end-to-end vehicle model in long-tail scenarios.

[0009] Secondly, large language models are prone to "semantic illusions," generating invalid instructions that violate physical principles and cannot be executed. When directly using large natural language models for scene strategy generation, the lack of direct understanding of 3D rigid body dynamics, physical laws, and the underlying simulation engine's rendering mechanism makes them highly susceptible to "semantic illusions that the physics engine cannot achieve." The generated text instructions may violate physical principles, or cause the underlying physics simulator to fail to render or even crash, greatly reducing the effectiveness and engineering executability of scene instructions.

[0010] Third, the "disorderly allocation of responsibility" between high- and low-level modules, coupled with noise gradients, interferes with the stability and accuracy of fine-tuning. When attempting to introduce multimodal graph-text consistency loss to constrain large language models, the final graph-text inconsistency may be caused by semantic illusions inherent in the large language model itself, or by insufficient mapping execution capabilities of the downstream parameter generation network (i.e., the generated physical parameter vectors deviate from the text expectations). Existing feedback mechanisms cannot effectively decouple this error source, leading to "disordered allocation of responsibility." During error backpropagation, non-target noise gradients are fully propagated back and erroneously interfere with the fine-tuning parameters of the large model, severely hindering the stable alignment of high-level semantic strategies. Summary of the Invention

[0011] To address the aforementioned technical problems, this invention provides a method for generating parameters for autonomous driving adversarial test scenarios based on user profiles and large models, comprising the following steps:

[0012] Step S1: Construct and maintain the dynamic capability profile matrix of the autonomous vehicle under test.

[0013] The dynamic capability profile matrix comprises a two-layer structure of a priori layer and an empirical layer; the priori layer stores the benchmark scores of the main vehicle in each dimension on the standard test set and the gradient sensitivity information obtained by performing static gradient sensitivity analysis on the main vehicle model;

[0014] The experience layer stores the performance score of the master vehicle in real time during the closed-loop adversarial simulation, the vectorized representation of failure cases, the gradient sensitivity of the network layer, and the capability improvement trajectory.

[0015] Step S2: Scene strategy text generation based on a large language model.

[0016] The dynamic capability profile matrix is ​​converted into a word sequence that the large language model can receive, and is used as the system's prompt words to be input into the large language model loaded with low-rank fine-tuning adapter parameters. The large language model then infers and outputs scenario strategy text targeting the main vehicle's weaknesses.

[0017] Step S3, parameter mapping and high-frequency fast-scale evolution.

[0018] The scene strategy text and the dynamic capability profile matrix are input into the scene generation network, which converts them into a continuous physical scene parameter vector. The physical scene parameter vector serves as a prerequisite input condition for driving the downstream simulation engine to perform scene rendering. The scene generation network adopts a fast time-scale parameter optimization mechanism, which optimizes high-frequency parameters with a learning rate greater than the master vehicle's learning rate and an inner loop update number greater than the master vehicle's update number.

[0019] Step S4: Align feedback based on direct preference optimization and text-image consistency strategy.

[0020] The feedback results of the closed-loop physical simulation are collected, the direct preference optimization loss and the cross-modal graph-text consistency loss are calculated, and the low-rank fine-tuning adapter parameters of the large language model are updated asynchronously using the direct preference optimization loss and the cross-modal graph-text consistency loss.

[0021] Specifically, when calculating the cross-modal image-text consistency loss, the scene policy text and the scene image actually rendered by the downstream simulation engine based on the physical scene parameter vector are first obtained. The scene policy text and the scene image are mapped to the same high-dimensional feature space and their similarity is calculated. The cross-modal image-text consistency loss is calculated based on the similarity.

[0022] Furthermore, the construction of the prior layer in step S1 includes:

[0023] At the initial moment of zero interaction in the system, the benchmark scores of the master vehicle in each dimension on the standard test set are extracted. For each violation or collision, the simulation video and log of that period are extracted and converted into natural language description using an open-source visual language model. Static gradient sensitivity analysis is performed on the master vehicle model. That is, without inputting simulation images, data with different semantic conditions are input into the master vehicle network, the gradient norm of each layer parameter is calculated, and feature dimensions whose gradient fluctuation exceeds the preset threshold are marked as unstable weaknesses of the master vehicle and written into the prior layer.

[0024] Furthermore, the construction and dynamic updating of the experience layer in step S1 includes the following sub-steps:

[0025] Step S11, smooth update of performance rating matrix: The system captures the objective test logs output by the underlying closed-loop simulator in real time, and automatically calculates and updates the historical success rate or normalized score of the main vehicle using the exponential moving average mechanism for different scenario label combinations.

[0026] Step S12, Vectorized representation of failure cases: When the main vehicle experiences a serious failure, the system automatically intercepts the policy text description and physical scene parameters of the scenario, uses a pre-trained text encoder to convert them into high-dimensional floating-point feature vectors, and persists them into the vector database.

[0027] Step S13, network layer gradient sensitivity extraction: When the main vehicle neural network performs backpropagation to update parameters, the system records the gradient norm generated by each core perception and decision module in real time. When a certain feature is detected to cause gradient fluctuations exceeding a preset threshold, it is marked as the main vehicle's cognitive blind spot and written into the profile.

[0028] Step S14, Capability Improvement Trajectory Tracking: The system dynamically caches the total performance score of the master vehicle in the past K consecutive training cycles, and constructs a one-dimensional time series change array as the capability improvement trajectory.

[0029] Furthermore, the dynamic capability profile matrix described in step S1 is updated using an exponential moving average mechanism. The weights of the prior layer decrease exponentially with the increase of training rounds, while the weights of the experience layer gradually increase with the increase of training rounds and approach 1, so that the capability profile smoothly transitions from static prior to real-time adversarial experience-driven.

[0030] Specifically, the system uses the exponential moving average mechanism for updates as follows:

[0031]

[0032] in, Indicates the first Dynamic capability profile matrix for each training round; Indicates the current round of adversarial training; This represents the prior layer weight. In the early stages of training, this weight is 1, and it increases with each training iteration. As the weight increases, the weight decreases exponentially. is the base of the natural logarithm; This represents the decay constant, which controls the speed of weight transfer. The prior layer represents the model's standard test set score and static gradient analysis results during the pre-training phase. It exists as the basis and decays over time. This represents the weight of the adversarial experience layer. As training progresses, this weight gradually increases and approaches 1, and the system will increasingly trust the real adversarial simulation experience. Indicates the smoothing coefficient; This represents the historical smoothing score accumulated in the previous round; Indicates the main vehicle is in the latest... The actual score for a single round of training;

[0033] As the number of training rounds increases, the weight of the prior layer gradually decays exponentially, while the weight of the experience layer gradually increases, allowing the profile to smoothly transition from static prior to real-time adversarial experience-driven.

[0034] Further, step S2 includes the following steps:

[0035] Step S21: Identify target weaknesses and construct structured cue words;

[0036] From the current dynamic capability profile matrix, at least one target weakness is selected according to a preset priority rule; the preset priority rule comprehensively considers the capability dimension score, the historical failure frequency corresponding to the capability dimension, the gradient sensitivity, and the capability improvement rate in the most recent rounds.

[0037] The numerical fields, category fields, and historical evidence fields in the dynamic capability profile matrix are serialized into structured text according to a preset field template, and the structured text is written into the capability profile data segment in the context of the prompt word.

[0038] Step S22: Load the low-rank fine-tuning adapter and form a task-adapted large language model;

[0039] Read the low-rank fine-tuning adapter parameters that match the base large language model identifier and the target network layer. For the target linear transformation layer in the base large language model, its base weight is denoted as W0. The low-rank fine-tuning adapter stores two low-rank matrices A and B. Then, the equivalent weight used during inference is expressed as W = W0 + (α / r)BA, where r represents the low-rank dimension and α represents the scaling factor. The incremental weights are dynamically superimposed on W0 during inference or merged into W0 before inference.

[0040] After the low-rank fine-tuning adapter is mounted or merged, the base large language model, the low-rank fine-tuning adapter, the word segmenter and the output layer together constitute the task-adapted large language model.

[0041] Step S23: Encode the structured prompt words and perform controlled autoregressive decoding;

[0042] The structured prompt words are converted into an input word sequence using a word segmenter that matches the base large language model. At the t-th generation position, the task-adapted large language model calculates the conditional probability of the next word based on the input word sequence and the first t-1 words that have been generated, and selects the next word according to the preset decoding parameters. The selected word is appended to the generated sequence and the above process is repeated until the end word is generated, the preset stop string is met, or the maximum output length is reached.

[0043] Step S24: Generate parsable scenario strategy text according to the preset scenario strategy mode;

[0044] Each candidate scenario strategy text adopts any parsable form of key-value pairs, JSON, YAML, or restricted natural language templates, and includes at least the target weakness identifier, scenario semantic category, road topology, environmental conditions, traffic participants and their relative relationships, key triggering events, expected failure mechanism, target difficulty level, and physical parameter constraint fields.

[0045] Step S25: Verify, repair, and select the candidate scenario strategy text;

[0046] The output validator executes the following in sequence: completeness check of required fields; semantic check of whether field values ​​belong to the preset autonomous driving scenario terminology set; constraint check of numerical units, upper and lower bounds and mutual exclusion conditions; and mappability check of each field to the input parameters of step S3.

[0047] For candidate scenario strategy texts that fail validation, the output validator generates structured error information containing missing fields, conflicting fields, or unmappable fields. This error information, along with the original structured prompts, is then input into the task adaptation large language model for a limited number of repair attempts. Candidate scenario strategy texts that still fail validation after a preset number of repair attempts are rejected, and the reason for rejection is recorded.

[0048] For multiple candidate scenario strategy texts that pass the verification, the weakness matching degree, physical mappability, internal consistency and novelty relative to historical strategies are calculated respectively, and the candidate quality score is obtained by weighted summation of each index; the scenario strategy text with the highest candidate quality score is selected, or sampling is performed according to the probability obtained by normalizing the candidate quality score, and the selected scenario strategy text and the corresponding dynamic capability profile matrix are jointly passed into step S3.

[0049] Further, in step S3, the scene generation network in the pre-parameter sampling and fast-scale adversarial module receives the scene strategy text and the dynamic capability profile matrix of the main vehicle, outputs a multi-dimensional Gaussian distribution, and samples to obtain a continuous physical scene parameter vector. The physical scene parameter vector includes several of the following: weather intensity, road curvature, initial coordinates of traffic participants, and triggering timing, which serve as prerequisite input conditions for driving the downstream simulation engine to perform scene rendering.

[0050] The scene generation network employs a fast time-scale parameter optimization mechanism: the learning rate of the scene generation network is set to be greater than that of the master vehicle, and in the inner loop parameter update, the scene generation network performs high-frequency gradient optimization with a higher update step count than the master vehicle; after receiving the scene policy text output by the large language model, the scene generation network performs local parameter optimization in the continuous physical scene parameter vector space based on the high-frequency update and large step size mechanism, and dynamically iterates and adjusts the parameters according to the real-time update feedback of the master vehicle's capability profile, so that the adversarial difficulty of the generated scene is continuously maintained near the current capability boundary of the master vehicle.

[0051] Furthermore, the method for constructing and calculating the direct preference optimization loss in step S4 is as follows:

[0052] Based on the generation results of the scene generation network and the objective feedback from the closed-loop physics engine, triples are automatically constructed, including prompt words representing the input, scene strategy text representing the winning preference, and scene strategy text representing the losing preference. If the physical scene parameter vector generated by the scene strategy text passes the difficulty validity check and causes the host vehicle to experience a high adversarial reward event in the physics simulation, then the text is marked as the winning preference scene strategy text. If the scene strategy text causes the generated parameters to violate the dynamic constraints and be intercepted by the hard constraint layer, or if the generated scene does not pose a threat to the host vehicle and falls into mode collapse, then the text is marked as the losing preference scene strategy text. The parameters of the large language model are updated by maximizing the generation probability ratio of the winning preference scene strategy text to the losing preference scene strategy text.

[0053] Furthermore, before calculating the cross-modal graph-text consistency loss in step S4, an error backpropagation gating step is also included: obtaining the state prediction value of the currently generated physical scene parameter vector through the difficulty evaluation head in the scene generation network, and quantifying the execution deviation rate between the underlying parameter generation distribution and the high-level scene strategy text; when the execution deviation rate is greater than or equal to a preset threshold, truncating or attenuating the upward propagation of the cross-modal graph-text consistency loss gradient in the current round; when the execution deviation rate is less than the preset threshold, backpropagating the error gradient generated by the cross-modal graph-text consistency loss to the large language model in its entirety.

[0054] Furthermore, the calculation method for cross-modal text-image consistency loss in step S4 is as follows:

[0055] The system obtains the scene policy text output by the large language model inference engine, and the multi-view image sequence or bird's-eye view actually rendered by the underlying conditional world model based on the text; the system calls the pre-trained cross-modal model, and through the frozen text encoder and image encoder, maps the text and the rendered screen to the same high-dimensional feature space to obtain text feature vectors and image feature vectors; the similarity is the cosine similarity between the text feature vector and the image feature vector, and the cross-modal image-text consistency loss is defined as 1 minus the cosine similarity.

[0056] Furthermore, in step S4, when acquiring the scene image, a spatiotemporal dual-dimensional intelligent downsampling and viewpoint scheduling mechanism is adopted: in the spatial dimension, based on the spatial orientation of semantic entities in the scene strategy text, the sensor channel with the most representativeness is dynamically selected, or the multimodal data is projected into a unified bird's-eye view semantic rendering image; in the temporal dimension, an event-driven sparse keyframe extraction strategy is adopted, based on the trigger timestamp in the physical scene parameter vector or the frame-by-frame difficulty curve output by the difficulty assessment header, the moment when the adversarial difficulty peaks is automatically locked, and only the frame at that moment or the extremely short window frame is extracted as the scene image.

[0057] Compared with the prior art, the present invention has the following advantages:

[0058] First, this invention overcomes the "cold start" challenge of adversarial training, improving the efficiency and accuracy of generating high-adversarial scenarios. Traditional testing methods treat the main vehicle as a complete black box, lacking effective attack experience in the early stages of adversarial training. They can only blindly sample or heuristically search within a vast parameter space, resulting in a massive waste of computing power on ineffective scenarios. This invention constructs a two-layer dynamic capability profile matrix comprising a prior layer and an experience layer: In the cold start phase, the prior layer extracts the main vehicle's score on the standard test set and, combined with white-box static gradient sensitivity analysis, accurately and quickly captures the cognitive blind spots and hidden weaknesses of the main vehicle's parameter structure regarding specific semantic features. During closed-loop adversarial training, the experience layer uses an exponential moving average mechanism to smoothly update performance scores in real time and utilizes a pre-trained text encoder to vectorize severe failure cases of the main vehicle into high-dimensional features, storing them in a vector database for accurate semantic clustering. This mechanism enables the testing agent to dynamically and accurately track the capability status of the tested main vehicle in real time, directly addressing the main vehicle's current systemic weaknesses when generating scenarios, significantly reducing the time and computing power costs of searching for long-tail scenarios.

[0059] Second, it achieves a perfect combination of macroscopic semantic diversity and high adversarial strength in the microscopic parameter space. Traditional parameter search algorithms lack high-level logical reasoning capabilities and are prone to pattern collapse in local optima, resulting in semantically homogeneous test scenarios. This invention adopts a fast and slow dual-timescale evolution mechanism in its architecture: the slow-timescale large language model reasoning module focuses on receiving the dynamic capability profile matrix, using its internalized general autonomous driving domain knowledge to perform high-level logical reasoning, and outputting macroscopic scenario strategy texts for attack categories; the fast-timescale pre-parameter sampling and fast-scale adversarial module utilizes a scenario generation network with a learning rate significantly greater than that of the main vehicle and a high-frequency internal loop update step count to perform high-frequency gradient local optimization in the continuous physical scenario parameter space. This high-low level decoupled fast-slow scale collaborative mechanism not only endows the system with the diversity and logic of macroscopic question generation, effectively preventing scenario homogenization, but also has the ability to perform fine-grained parameter micro-management in the underlying continuous physical space.

[0060] Third, it strongly constrains semantic illusions, ensuring that the generated high-level instructions are strictly aligned with the underlying physical feasible domain and executability. When an unaligned large language model is directly applied to scene generation, the lack of direct understanding of the 3D physical world rendering mechanism by the large language model easily leads to "semantic illusions that the physics engine cannot achieve," causing the underlying simulator to crash. This invention introduces a dual-feedback fine-tuning mechanism based on direct preference optimization and image-text consistency, directly affecting the weight parameters of the low-rank fine-tuning adapter of the large language model. The direct preference optimization loss utilizes the objective feedback from the closed-loop physics engine to automatically construct triplet positive and negative preference data, driving the fine-tuned large language model strategy to monotonically approximate the true reward upper limit of the physical environment; the cross-modal image-text consistency loss utilizes a pre-trained contrastive language-image pre-trained model to calculate the cosine similarity between the text description output by the large language model and the image sequence actually rendered by the underlying world model in the same high-dimensional feature space. The joint optimization of the dual feedback forms a strong engineering constraint, effectively suppressing the physical infeasibility bias that is prone to occur in scene generation by the natural language large model.

[0061] Fourth, the precise decoupling of system responsibility allocation significantly enhances the stability and accuracy of large-scale model fine-tuning. In joint fine-tuning, inconsistencies between the final rendered image and the target text of the large model may be attributed to semantic illusions inherent in the large language model itself, or to insufficient mapping execution capabilities of the downstream parameter generation network. Traditional feedback mechanisms cannot distinguish this error source, easily leading to misallocation of responsibility and noise gradient contamination. This invention innovatively introduces an error backpropagation gating mechanism based on execution confidence: when the execution deviation rate is greater than or equal to a preset threshold, the mismatch is determined to originate primarily from the underlying execution end. The gating mechanism automatically activates and truncates or attenuates the upward propagation of the cross-modal image-text consistency loss gradient in the current round, isolating the error at the execution end; only when the execution deviation rate is less than the preset threshold and the underlying layer has accurately completed the physical space mapping is the gating fully activated, propagating the gradient back to the large language model in its entirety. This gating mechanism achieves precise decoupling of system responsibility between high- and low-level modules, effectively filtering local noise at the execution end and improving the stability of joint system training.

[0062] Fifth, this invention significantly optimizes the multimodal computational load and improves the overall system efficiency. Calculating the cross-modal contrastive language-image pre-training loss between continuous high-dimensional multimodal sensor video streams and large-model text typically requires extremely high computing power. This invention introduces a spatiotemporal dual-dimensional intelligent downsampling and viewpoint scheduling mechanism when calculating the consistency loss: in the spatial dimension, it dynamically selects the most representative sensor channel based on the spatial orientation of semantic entities in the strategy text, or uniformly projects it as a global bird's-eye view semantic rendering image for comparison; in the temporal dimension, it adopts an event-driven sparse keyframe extraction strategy, automatically locking the critical moment when the adversarial difficulty peaks based on the parameter trigger timestamp or the frame-by-frame difficulty curve output by the difficulty assessment header, and only extracting that instantaneous frame or a very short window frame for calculation. This optimization mechanism significantly reduces the system's computational load and memory consumption while fully preserving the core adversarial features and without affecting the accuracy of the consistency loss calculation. Attached Figure Description

[0063] Figure 1 This is a logical diagram of an autonomous driving test scenario generation system provided in an embodiment of the present invention.

[0064] Figure 2 This is a flowchart illustrating the method for generating parameters for autonomous driving test scenarios provided in an embodiment of the present invention. Detailed Implementation

[0065] The technical solutions in the embodiments of the present invention will now be clearly and completely described with reference to the accompanying drawings. Obviously, the described embodiments are merely some, not all, of the embodiments of the present invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention.

[0066] This embodiment provides an autonomous driving test scenario generation system, which is deployed on the test agent terminal, such as... Figure 1 As shown, the system mainly includes the following core modules:

[0067] (1) Capability Profile Construction Module

[0068] This module is responsible for collecting historical static evaluation results, network layer gradient sensitivity, and dynamic simulation logs of the autonomous vehicle under test, and constructing and maintaining a two-layer dynamic capability profile matrix containing a "prior layer" and an "empirical layer". .

[0069] (2) Large Language Model Reasoning Module

[0070] This module contains a large language model (LLM) loaded with parameters from a low-rank adaptation (LoRA). It is responsible for receiving a dynamic capability profile matrix, performing natural language inference, and outputting scenario-based strategy text targeting the systemic weaknesses of the host vehicle. In this embodiment, the large language model can be an open-source large language model (such as LLaMA, Qwen, etc.), and the parameters can be efficiently fine-tuned using low-rank fine-tuning adapter technology.

[0071] (3) Pre-parameter sampling and fast-scale adversarial module

[0072] This module internally deploys a scenario generation network (i.e., a test agent policy network). This module receives the scenario policy text. and the main vehicle dynamic capability profile matrix The scene is transformed into a continuous physical scene parameter vector through a scene generation network. The vector It includes structured definitions of the environment, roads, and traffic participants, serving as prerequisite inputs for driving high-fidelity sensor data rendering in the downstream world model. This module incorporates a fast timescale engine, utilizing a large learning rate and high-frequency inner loop update steps to rapidly explore the parameter space.

[0073] (4) Alignment and fine-tuning feedback module

[0074] This module is responsible for collecting feedback results from closed-loop physical simulations, calculating the Direct Preference Optimization (DPO) loss and the Contrastive Language-Image Pretraining (CLIP) text-image consistency loss, and using these to asynchronously update the low-rank fine-tuning adapter parameters of the large language model.

[0075] like Figure 2 As shown, this embodiment provides a method for generating parameters for autonomous driving adversarial test scenarios based on profiles and large models. Running on the aforementioned autonomous driving test scenario generation system, the method includes the following steps:

[0076] Step S1: Construct and maintain the dynamic capability profile matrix of the autonomous driving master vehicle under test (hereinafter referred to as "master vehicle").

[0077] The dynamic capability profile matrix comprises a two-layer structure of a priori layer and an empirical layer; the priori layer stores the benchmark scores of the main vehicle in each dimension on the standard test set and the gradient sensitivity information obtained by performing static gradient sensitivity analysis on the main vehicle model;

[0078] The experience layer stores the performance score of the master vehicle in real time during the closed-loop adversarial simulation, the vectorized representation of failure cases, the gradient sensitivity of the network layer, and the capability improvement trajectory.

[0079] Specifically, the construction of the prior layer includes:

[0080] At the initial moment of zero interaction in the system, the benchmark scores of the master vehicle in each dimension on the standard test set (such as the completion rate in sunny weather and the success rate of obstacle avoidance) are extracted. For each violation or collision, the simulation video and log of that period are extracted and converted into natural language description using an open-source visual language model. Static gradient sensitivity analysis is performed on the master vehicle model. That is, without inputting simulation images, data with different semantic conditions are input into the master vehicle network, the gradient norm of each layer parameter is calculated, and feature dimensions with gradient fluctuations exceeding the preset threshold (such as "night" and "backlight") are marked as unstable weaknesses of the master vehicle and written into the prior layer.

[0081] As the closed-loop adversarial simulation continues, the system not only intercepts failed cases of collisions or violations by the main vehicle in real time, but also automatically extracts multi-dimensional data from objective physical environment feedback and the internal state of the main vehicle's neural network. Therefore, the construction and dynamic updating of the experience layer includes the following sub-steps:

[0082] Step S11, smooth update of performance rating matrix: The system captures the objective test logs (including indicators such as collision, violation, and time consumption) output by the underlying closed-loop simulator in real time, and automatically calculates and updates the historical success rate or normalized score of the main vehicle using the exponential moving average mechanism for different scenario label combinations.

[0083] Step S12, Vectorization of Failure Cases: When the main vehicle experiences a serious failure (such as a collision, running a red light, or deadlock due to lane departure), the system automatically intercepts the policy text description (i.e., the original policy generated by the large language model) and physical scene parameters of the scenario. It then uses an internally frozen lightweight pre-trained text encoder (such as Sentence-BERT) to transform it into a high-dimensional floating-point feature vector (768-dimensional array) and persists it in the vector database. This mechanism transforms discrete text into continuous mathematical representations, enabling the large language model to perform semantic clustering based on vector similarity, thereby accurately discovering and locating the systemic weaknesses of the main vehicle.

[0084] Step S13, network layer gradient sensitivity extraction: When the main vehicle neural network performs backpropagation to update parameters, the system records the gradient norm generated by each core perception and decision-making module (such as BEV encoder, temporal fusion layer, trajectory planning head) in real time. When a certain feature (such as "night" or "backlight") is detected to cause gradient fluctuations exceeding the preset threshold, it is marked as the main vehicle's cognitive blind spot and written into the profile; thus realizing white-box monitoring of the internal state of the main vehicle model.

[0085] Step S14, Capability Improvement Trajectory Tracking: The system dynamically caches the total performance scores of the main vehicle over the past K consecutive training cycles, constructing a one-dimensional temporal change array as the capability improvement trajectory. This trajectory flow is fed in as auxiliary features, giving the large language model temporal inference capabilities, enabling it to clearly determine whether the main vehicle is currently in a "steady improvement period" or has fallen into a "capability bottleneck period," thereby adaptively adjusting the difficulty and direction of the questions.

[0086] Furthermore, the dynamic capability profile matrix is ​​updated using an exponential moving average mechanism. The weights of the prior layer decrease exponentially with the increase of training rounds, while the weights of the experience layer gradually increase with the increase of training rounds and approach 1, so that the capability profile smoothly transitions from static prior to real-time adversarial experience-driven.

[0087] Specifically, the system uses the exponential moving average mechanism for updates as follows:

[0088]

[0089] in, Indicates the first Dynamic capability profile matrix for each training round; Indicates the current adversarial training loop. This represents the weights of the prior layer, in the early stages of training ( The weight is 1, and it increases with the number of training rounds. As the weight increases, the weight decreases exponentially. is the base of the natural logarithm; This represents the decay constant, which controls the speed of weight transfer. The prior layer represents the model's standard test set score and static gradient analysis results during the pre-training phase. It exists as the basis and decays over time. This represents the weight of the adversarial experience layer. As training progresses, this weight gradually increases and approaches 1, and the system will increasingly trust the real adversarial simulation experience. This represents the smoothing coefficient, typically ranging from 0.9 to 0.99. In this embodiment, the system is configured as follows: This means that 90% relies on historical accumulation, and 10% is affected by the latest single result; This represents the historical smoothing score accumulated in the previous round (old profile). Indicates the main vehicle is in the latest... The actual score of a single round during training (i.e., the objective reward from environmental feedback);

[0090] As the number of training rounds increases, the weight of the prior layer gradually decays exponentially, while the weight of the experience layer gradually increases, allowing the profile to smoothly transition from static prior to real-time adversarial experience-driven.

[0091] Step S2: Scene strategy text generation based on a large language model.

[0092] The dynamic capability profile matrix is ​​converted into a word sequence that the large language model can receive, and is used as the system's prompt words to be input into the large language model loaded with low-rank fine-tuning adapter parameters. The large language model then infers and outputs scenario strategy text targeting the main vehicle's weaknesses.

[0093] In this embodiment, step S2 does not directly input the numerical matrix object in memory into the large language model. Instead, it converts the dynamic capability profile matrix into a lexical sequence that the large language model can accept, and uses a task-adaptive large language model loaded with a low-rank fine-tuning adapter to generate and verify scenario strategy text. Step S2 specifically includes steps S21 to S25:

[0094] Step S21: Identify target weaknesses and construct structured cue words.

[0095] From the current dynamic capability profile matrix, at least one target weakness is selected according to a preset priority rule. The preset priority rule comprehensively considers the capability dimension score, the historical failure frequency corresponding to the capability dimension, the gradient sensitivity, and the capability improvement rate in the most recent rounds. Dimensions with lower capability scores, higher failure frequencies, higher gradient sensitivity, or stagnant capability improvement have higher priority.

[0096] The numerical fields, category fields, and historical evidence fields in the dynamic capability profile matrix are serialized into structured text according to a preset field template, and the structured text is written into the capability profile data segment in the context of the prompt words. The structured prompt words include at least: a task instruction segment for specifying the model role and task objective; a capability profile data segment containing the main vehicle version, target weakness identifier, capability score, gradient sensitivity, historical failure evidence, and capability change trend; a scene constraint segment containing allowed roads, weather, traffic participants, event triggering conditions, and prohibited combinations; and an output format segment specifying the required fields, field types, and output order.

[0097] Step S22: Load the low-rank fine-tuning adapter and form a task-adapted large language model.

[0098] Read the low-rank fine-tuning adapter parameters that match the base large language model identifier and the target network layer. For the target linear transformation layer in the base large language model, its base weight is denoted as W0. The low-rank fine-tuning adapter stores two low-rank matrices A and B. Then, the equivalent weight used during inference is expressed as W = W0 + (α / r)BA, where r represents the low-rank dimension and α represents the scaling factor. The incremental weights are dynamically superimposed on W0 during inference or merged into W0 before inference.

[0099] The low-rank fine-tuning adapter itself does not perform word segmentation, next word prediction, or text output; its role is to modify the forward computation results of the target network layer of the base large language model. After the low-rank fine-tuning adapter is attached or merged, the base large language model, the low-rank fine-tuning adapter, the word segmenter, and the output layer together constitute the task-adapted large language model. During one inference step S2, the base weights W0 and the adapter matrices A and B remain fixed, and the low-rank fine-tuning adapter parameters of the large language model updated in step S4 are loaded in subsequent inference rounds.

[0100] Step S23: Encode the structured prompt words and perform controlled autoregressive decoding.

[0101] The structured prompt words are converted into an input word sequence using a word segmenter matched with the base large language model. At the t-th generation position, the task-adapted large language model calculates the conditional probability of the next word based on the input word sequence and the first t-1 words already generated, and selects the next word according to preset decoding parameters. The selected word is appended to the generated sequence and the above process is repeated until the end word is generated, the preset stop string is met, or the maximum output length is reached. The preset decoding parameters may include temperature parameters, kernel sampling threshold top-p, number of candidate words top-k, maximum output length, and random seed. Thus, the complete large language model after loading the low-rank fine-tuning adapter can directly generate one or more candidate scene policy texts without the need for additional text generation models.

[0102] Step S24: Generate parsable scenario strategy text according to the preset scenario strategy mode.

[0103] Each candidate scenario strategy text adopts any parsable form of key-value pairs, JSON, YAML, or restricted natural language templates, and at least includes target weakness identifiers, scenario semantic categories, road topology, environmental conditions, traffic participants and their relative relationships, key triggering events, expected failure mechanisms, target difficulty level, and physical parameter constraint fields. The physical parameter constraint fields are used to give parameter categories, allowable intervals, or semantic levels, rather than directly determining all continuous physical parameters in step S2. The specific continuous physical scenario parameters are mapped and optimized by the scenario generation network in step S3.

[0104] For example, regarding the target weakness in the capability profile of "weak left-turn decision-making ability under fog conditions without protection", the candidate scenario strategy text can be described as follows: the target weakness is left-turn decision-making without protection in fog, the scenario semantic category is intersection conflict, the road topology is a four-armed intersection, the environmental condition is moderate fog concentration, the key traffic participant is oncoming approaching vehicles, the triggering event is the main vehicle entering the left-turn conflict zone, the expected failure mechanism is the delay in judging the right-of-way timing, the target difficulty is medium to high, and allowable ranges are given for visibility, oncoming vehicle speed, and initial distance.

[0105] Step S25: Verify, repair, and select the candidate scenario strategy text.

[0106] The output validator executes the following in sequence: completeness check of required fields; semantic check of whether field values ​​belong to the preset autonomous driving scenario terminology set; constraint check of numerical units, upper and lower bounds and mutual exclusion conditions; and mappability check of whether each field can be mapped to the input parameters of step S3. This check is a format and logic check of the generation front end and does not replace the verification of the scene attack effect in step S3 and subsequent simulation.

[0107] For candidate scenario strategy texts that fail validation, the output validator generates structured error information containing missing fields, conflicting fields, or unmappable fields. This error information, along with the original structured prompts, is then input into the task adaptation large language model for a limited number of repair attempts. Candidate scenario strategy texts that still fail validation after a preset number of repair attempts are rejected, and the reason for rejection is recorded.

[0108] For multiple candidate scenario strategy texts that pass the verification, the weakness matching degree, physical mappability, internal consistency and novelty relative to historical strategies are calculated respectively, and the candidate quality score is obtained by weighted summation of each index; the scenario strategy text with the highest candidate quality score is selected, or sampling is performed according to the probability obtained by normalizing the candidate quality score, and the selected scenario strategy text and the corresponding dynamic capability profile matrix are jointly passed into step S3.

[0109] Step S3, parameter mapping and high-frequency fast-scale evolution.

[0110] The scene strategy text and the dynamic capability profile matrix are input into the scene generation network, which converts them into a continuous physical scene parameter vector. The physical scene parameter vector serves as a prerequisite input condition for driving the downstream simulation engine to perform scene rendering. The scene generation network adopts a fast time-scale parameter optimization mechanism, which optimizes high-frequency parameters with a learning rate greater than the master vehicle's learning rate and an inner loop update number greater than the master vehicle's update number.

[0111] Specifically, step S3 in this embodiment includes the following steps:

[0112] Step S31, configuration parameter generation:

[0113] The scene generation network within the pre-parameter sampling and fast-scale adversarial module receives scene policy text. The system also generates a dynamic capability profile matrix for the main vehicle, outputting a multidimensional Gaussian distribution and sampling continuous physical scene parameter vectors. (Covering weather intensity, road curvature, NPC initial coordinates, and trigger timing, etc.); At this point, the test agent has completed the core tasks of "question generation" and "data preparation," and this parameter vector... With text instructions This will also serve as a prerequisite input condition for driving the downstream conditional world model to perform scene rendering, guiding it to complete the final high-fidelity multimodal sensor data rendering.

[0114] Step S32, Fast-scale exploration (core mechanism):

[0115] The policy network of the test agent is given a "fast time scale" characteristic during optimization; the learning rate of the test agent is set. The learning rate is significantly greater than that of the master vehicle, and the test agent's execution is tested during the internal loop parameter update. High-frequency gradient optimization of steps (satisfying) (Greater than the number of steps updated by the main vehicle). In receiving the scene strategy text output by the large language model. Subsequently, the test agent, based on a high-frequency update and large step size mechanism, updates the parameter vectors in a continuous physical scene. Within the space, local parameters are optimized; based on the real-time update feedback of the master vehicle's dynamic capability profile matrix, the distance or environmental parameters of traffic participants (such as fog concentration) are dynamically and iteratively adjusted to keep the adversarial difficulty of the generated scenario close to the current capability boundary of the master vehicle, thereby providing effective long-tail adversarial samples for system stability.

[0116] Step S4: Align feedback based on direct preference optimization and text-image consistency strategy.

[0117] The feedback results of the closed-loop physical simulation are collected, the direct preference optimization loss and the cross-modal graph-text consistency loss are calculated, and the low-rank fine-tuning adapter parameters of the large language model are updated asynchronously using the direct preference optimization loss and the cross-modal graph-text consistency loss.

[0118] Specifically, when calculating the cross-modal image-text consistency loss, the scene policy text and the scene image actually rendered by the downstream simulation engine based on the physical scene parameter vector are first obtained. The scene policy text and the scene image are mapped to the same high-dimensional feature space and their similarity is calculated. The cross-modal image-text consistency loss is calculated based on the similarity.

[0119] Specifically, to prevent large language models from generating random questions (such as generating scenarios that violate physical principles or cannot be rendered by the underlying simulator), this invention introduces a dual feedback fine-tuning mechanism to update the low-rank fine-tuning adapter parameters of the large language model, including the following steps:

[0120] Step S41, Construction and calculation of direct preference optimization loss:

[0121] set up The input prompts (including the vehicle's dynamic capability profile matrix) The system automatically constructs triples based on the generation status of the downstream scene's network and the objective feedback from the closed-loop physics engine. , These represent the strategy texts for scenarios where the preference wins and the strategy texts for scenarios where the preference loses, respectively. If the strategy text generated by the large language model successfully drives the parameter vector generated downstream to pass the internal difficulty validity review (without deviating from the course objectives), and leads to high-adversarial reward events such as collisions or violations by the main vehicle in actual physical simulations, then this strategy text is marked as... If the policy text instructions generated by the large language model are too extreme, causing the downstream generated parameter vectors to violate dynamics and be intercepted by the system's hard constraint layer (such as the anti-collision rejection module), or if the generated scenario fails to pose a substantial threat to the main vehicle and falls into mode collapse (diversity score of zero), then the policy text is marked as... By maximizing the probability ratio between the two factors through a direct preference optimization algorithm, large language models are made to tend to generate feasible and lethal scenario text.

[0122] The formula for calculating the direct preference optimization loss is:

[0123]

[0124] in, This represents the direct preference optimization loss. For the large language model currently being trained with low-rank fine-tuning adapter parameters; The original large language model is completely frozen (used as a baseline to prevent catastrophic forgetting of the model). Indicates from preference dataset Triples obtained from sampling The expected value of the corresponding loss is calculated; in actual training, it can be approximated by the average value of the loss of all triples in a batch. For the Sigmoid function; Temperature hyperparameters are used to control the intensity of preference and adjust the model's sensitivity to the probability difference between texts that win and lose in preference. This indicates that the large language model currently being trained is performing well given the input prompt word. Under the given conditions, the conditional probability distribution formed for all possible generated scenario policy texts, where the parameters are... Including the low-rank fine-tuning adapter parameters that are currently to be optimized. The reference large language model indicates that the parameters remain frozen given the input prompt word. Under the given conditions, the conditional probability distribution formed for all possible generated scenario policy texts is used to constrain the current model from deviating excessively from the original model during the preference optimization process. and These represent the complete text generated by the current model. and The conditional probability. The advantage of this formula is that it allows us to directly maximize the conditional probability without needing to train a large reward model. and The generation probability ratio makes the fine-tuned large language model strategy monotonically approximate the upper limit of the real reward in the physical environment.

[0125] Step S42, Cross-modal graph-text consistency loss Construction and calculation of error feedback gating:

[0126] Because large language models lack direct understanding of the rendering mechanism of the three-dimensional physical world, they are prone to "semantic illusions that the physics engine cannot achieve" (such as describing "dense fog as solid as a solid wall," which causes the underlying engine to be unable to render). To eliminate this gap between high-level semantics and low-level physics, this invention introduces a text-image consistency verification module.

[0127] Specifically, the scene strategy text output by the large language model inferencer is obtained. And the multi-view image sequence or bird's-eye view (BEV) actually rendered by the underlying conditional world model (or CARLA renderer) based on the text. The system calls a pre-trained cross-modal model (contrastive language-image pre-trained model) through a frozen text encoder. and image encoder The text and the rendered image are mapped to the same high-dimensional feature space respectively, resulting in text feature vectors. and image feature vectors .

[0128] It should be noted that, due to the original representation output by the underlying conditional world model... To address the challenge of constructing a high-dimensional continuous video stream containing multiple camera views and point clouds, and considering that the computational space of a contrastive language-image pre-trained model is typically limited to a single frame of standard RGB image, this system introduces a spatiotemporal dual-dimensional intelligent downsampling and viewpoint scheduling mechanism to ensure... Efficiency and accuracy of calculations:

[0129] In the spatial dimension (viewpoint scheduling and projection), the system relies on scene strategy text. The system dynamically selects the most representative sensor channel based on the spatial orientation of semantic entities (e.g., extracting the rear-view camera for "rear-end collision" and the front-view camera for "oncoming traffic"). As a preferred embodiment, the system can also project multimodal data into a unified bird's-eye view (BEV) semantic rendering image, using the global RGB image as input for cross-modal image-text consistency comparison.

[0130] In the time dimension (event-driven keyframe extraction), to avoid computational overload caused by full computation of long video sequences, the system adopts an event-driven sparse keyframe extraction strategy; the system does not calculate the average loss over the entire time period, but instead uses the generated scene parameter vector. The system automatically identifies critical moments when the adversarial difficulty peaks, based on the trigger timestamps in the system or the frame-by-frame difficulty curve output from the difficulty assessment header within the policy network. The system then extracts only a single frame or a very short window frame at that peak moment and performs feature space similarity calculations with the policy text.

[0131] Text feature vector and image feature vectors Cosine similarity in space The calculation formula is as follows:

[0132]

[0133] Cross-modal image-text consistency loss is defined as:

[0134]

[0135] However, given that the final inconsistency between the text and the image may be caused by semantic illusions from the large language model, or by insufficient execution capabilities of the downstream parameter generation network (i.e., the generated physical scene parameter vectors), the issue could be misleading. To address the issue of misallocation of responsibility caused by deviations from expected text instructions, this system introduces an error feedback gating mechanism based on execution confidence.

[0136] Calculating cross-modal image-text consistency loss Previously, the system obtained the current generation parameters by testing the difficulty evaluation head within the agent's policy network. The state prediction value is used to quantitatively evaluate the execution deviation rate between the low-level parameter generation distribution and the high-level target text instruction, and gradient gating decisions are performed based on this deviation rate:

[0137] When the execution deviation rate is greater than or equal to a preset threshold, the system determines that the current multimodal representation mismatch mainly stems from the execution-end mapping error of the parameter generation network. At this point, the system activates a gradient gating mechanism to truncate (or reduce) the current round by a set decay coefficient. Gradient uppropagation. The system only retains the policy gradient for local updates of the parameter generation network, thereby effectively isolating error propagation paths and preventing non-target noise gradients from erroneously interfering with the fine-tuning of parameters in large language models.

[0138] When the execution deviation rate is less than a preset threshold, the system determines that the parameter generation network has accurately completed the physical space mapping of the high-level instructions. The inconsistency between the graphical and textual representations is essentially due to the large language model outputting invalid semantic strategies that exceed the feasible rendering domain of the physical simulation engine. At this time, gradient gating is in the on state, and the system will... The generated error gradient is backpropagated in its entirety to the large language model, driving the low-rank fine-tuning adapter parameters to make directional corrections, thereby eliminating semantic bias in the high-level decision-making module.

[0139] If the text generated by the large language model exceeds the expression boundary of the underlying physics engine, resulting in the text and image being completely unrelated, the two vectors tend to be orthogonal, with extremely low cosine similarity, thus generating a huge consistency penalty error. The gradient of this error will be directly backpropagated and update the low-rank fine-tuning adapter parameters of the large language model.

[0140] By jointly optimizing the aforementioned dual loss function, the large language model of the test agent achieves multi-dimensional policy alignment: on the one hand, the loss is optimized by direct preferences. The driving model generates highly adversarial scenario strategies targeting the vulnerable boundaries of the main vehicle; on the other hand, it utilizes cross-modal graph-text consistency loss. The constraint model is strictly aligned with the physical and rendering feasible domains of the underlying closed-loop simulation engine. This joint mechanism effectively suppresses the physical infeasibility bias and semantic illusion that are prone to occur in the generation of autonomous driving test scenarios by large natural language models, and significantly improves the effectiveness and engineering executability of scene instructions.

Claims

1. A method for generating parameters for autonomous driving adversarial test scenarios based on user profiles and large models, characterized in that, Includes the following steps: Step S1: Construct a dynamic capability profile matrix, which includes a two-layer structure of prior layer and experience layer; The prior layer stores the benchmark scores of the master vehicle in each dimension on the standard test set and the gradient sensitivity information obtained by static gradient sensitivity analysis of the master vehicle model; the experience layer stores the performance scores of the master vehicle collected in real time during the closed-loop adversarial simulation, the vectorized representation of failure cases, the gradient sensitivity of the network layer, and the capability improvement trajectory. Step S2: Input the dynamic capability profile matrix as the system prompt words into the large language model loaded with low-rank fine-tuning adapter parameters, and output the scenario strategy text targeting the main vehicle's weaknesses by the large language model inference. Step S3: Input the scene strategy text and the dynamic capability profile matrix into the scene generation network, which will then convert them into a continuous physical scene parameter vector. This vector will serve as a prerequisite input for driving the downstream simulation engine to render the scene. The scene generation network will employ a fast time-scale parameter optimization mechanism, using a learning rate greater than the master vehicle's learning rate and an inner loop update number greater than the master vehicle's update number for high-frequency parameter optimization. Step S4: Collect feedback results from closed-loop physical simulation, calculate direct preference optimization loss and cross-modal graph-text consistency loss, and use direct preference optimization loss and cross-modal graph-text consistency loss to asynchronously update the low-rank fine-tuning adapter parameters of the large language model. When calculating the cross-modal image-text consistency loss, the scene policy text and the scene image actually rendered by the downstream simulation engine based on the physical scene parameter vector are first obtained. The scene policy text and the scene image are mapped to the same high-dimensional feature space and their similarity is calculated. The cross-modal image-text consistency loss is calculated based on the similarity.

2. The method for generating parameters for autonomous driving adversarial test scenarios based on profiles and large models according to claim 1, characterized in that: The prior layer described in step S1 is constructed as follows: At the initial moment of zero interaction in the system, the benchmark scores of the master vehicle in each dimension on the standard test set are extracted. For each violation or collision, the simulation video and log of that period are extracted and converted into natural language description using an open-source visual language model. Static gradient sensitivity analysis is performed on the master vehicle model. That is, without inputting simulation images, data with different semantic conditions are input into the master vehicle network, the gradient norm of each layer parameter is calculated, and feature dimensions whose gradient fluctuation exceeds the preset threshold are marked as unstable weaknesses of the master vehicle and written into the prior layer.

3. The method for generating parameters for autonomous driving adversarial test scenarios based on profiles and large models according to claim 1, characterized in that: The construction and dynamic updating of the experience layer in step S1 includes the following sub-steps: Step S11, smooth update of performance rating matrix: The system captures the objective test logs output by the underlying closed-loop simulator in real time, and automatically calculates and updates the historical success rate or normalized score of the main vehicle using the exponential moving average mechanism for different scenario label combinations. Step S12, Vectorized representation of failure cases: When the main vehicle experiences a serious failure, the system automatically intercepts the policy text description and physical scene parameters of the scenario, uses a pre-trained text encoder to convert them into high-dimensional floating-point feature vectors, and persists them into the vector database. Step S13, network layer gradient sensitivity extraction: When the main vehicle neural network performs backpropagation to update parameters, the system records the gradient norm generated by each core perception and decision module in real time. When a certain feature is detected to cause gradient fluctuations exceeding a preset threshold, it is marked as the main vehicle's cognitive blind spot and written into the profile. Step S14, Capability Improvement Trajectory Tracking: The system dynamically caches the total performance score of the master vehicle in the past K consecutive training cycles, and constructs a one-dimensional time series change array as the capability improvement trajectory.

4. The method for generating parameters for autonomous driving adversarial test scenarios based on profiles and large models according to claim 1, characterized in that: The dynamic capability profile matrix described in step S1 is updated using an exponential moving average mechanism. The weights of the prior layer decrease exponentially with the increase of training rounds, while the weights of the experience layer gradually increase with the increase of training rounds and approach 1, so that the capability profile smoothly transitions from static prior to real-time adversarial experience-driven.

5. The method for generating parameters for autonomous driving adversarial test scenarios based on profiles and large models according to claim 4, characterized in that: The system uses the exponential moving average mechanism for updates as follows: in, Indicates the first Dynamic capability profile matrix for each training round; Indicates the current round of adversarial training; This represents the prior layer weight. In the early stages of training, this weight is 1, and it increases with each training iteration. As the weight increases, the weight decreases exponentially. is the base of the natural logarithm; This represents the decay constant, which controls the speed of weight transfer. The prior layer represents the model's standard test set score and static gradient analysis results during the pre-training phase. It exists as the basis and decays over time. This represents the weight of the adversarial experience layer. As training progresses, this weight gradually increases and approaches 1, and the system will increasingly trust the real adversarial simulation experience. Indicates the smoothing coefficient; This represents the historical smoothing score accumulated in the previous round; Indicates the main vehicle is in the latest... The actual score for a single round of training; As the number of training rounds increases, the weight of the prior layer gradually decays exponentially, while the weight of the experience layer gradually increases, allowing the profile to smoothly transition from static prior to real-time adversarial experience-driven.

6. The method for generating parameters for autonomous driving adversarial test scenarios based on profiles and large models according to claim 1, characterized in that: Step S2 includes the following steps: Step S21: Select at least one target weakness from the current dynamic capability profile matrix according to a preset priority rule; the preset priority rule comprehensively considers the capability dimension score, the historical failure frequency corresponding to the capability dimension, the gradient sensitivity, and the capability improvement rate in the most recent rounds. The numerical fields, category fields, and historical evidence fields in the dynamic capability profile matrix are serialized into structured text according to a preset field template, and the structured text is written into the capability profile data segment in the context of the prompt word. Step S22: Read the low-rank fine-tuning adapter parameters that match the base large language model identifier and the target network layer. For the target linear transformation layer in the base large language model, its base weight is denoted as W0. The low-rank fine-tuning adapter stores two low-rank matrices A and B. The equivalent weight used during inference is expressed as W = W0 + (α / r)BA, where r represents the low-rank dimension and α represents the scaling factor. The incremental weights are dynamically superimposed on W0 during inference or merged into W0 before inference. After the low-rank fine-tuning adapter is mounted or merged, the base large language model, the low-rank fine-tuning adapter, the word segmenter and the output layer together constitute the task-adapted large language model. Step S23: The prompt word is converted into an input word sequence using a word segmenter that matches the base large language model; at the t-th generation position, the task-adapted large language model calculates the conditional probability of the next word based on the input word sequence and the first t-1 words that have been generated, and selects the next word according to the preset decoding parameters; the selected word is appended to the generated sequence and the above process is repeated until the end word is generated, the preset stop string is met, or the maximum output length is reached; Step S24: Each candidate scenario strategy text adopts any parsable form of key-value pairs, JSON, YAML or restricted natural language template, and includes at least the target weakness identifier, scenario semantic category, road topology, environmental conditions, traffic participants and their relative relationships, key triggering events, expected failure mechanism, target difficulty level and physical parameter constraint fields. Step S25, the output validator performs the following in sequence: completeness check of required fields; semantic check of whether field values ​​belong to the preset autonomous driving scenario terminology set; constraint check of numerical units, upper and lower bounds and mutual exclusion conditions; and mappability check of each field to the input parameters in step S3. For candidate scenario strategy texts that fail validation, the output validator generates structured error information containing missing fields, conflicting fields, or unmappable fields, and inputs the error information and the original prompt words into the task adaptation large language model for limited repair generation; candidate scenario strategy texts that still fail validation after reaching the preset number of repairs are rejected, and the reason for rejection is recorded. For multiple candidate scenario strategy texts that pass the verification, the weakness matching degree, physical mappability, internal consistency and novelty relative to historical strategies are calculated respectively, and the candidate quality score is obtained by weighted summation of each index; the scenario strategy text with the highest candidate quality score is selected, or sampling is performed according to the probability obtained by normalizing the candidate quality score, and the selected scenario strategy text and the corresponding dynamic capability profile matrix are jointly passed into step S3.

7. The method for generating parameters for autonomous driving adversarial test scenarios based on profiles and large models according to claim 1, characterized in that: In step S3, the scene generation network in the pre-parameter sampling and fast-scale adversarial module receives the scene strategy text and the dynamic capability profile matrix of the main vehicle, outputs a multi-dimensional Gaussian distribution, and samples a continuous physical scene parameter vector. The physical scene parameter vector includes several of the following: weather intensity, road curvature, initial coordinates of traffic participants, and triggering timing, which serve as prerequisite input conditions for driving the downstream simulation engine to perform scene rendering. The scene generation network employs a fast time-scale parameter optimization mechanism: the learning rate of the scene generation network is set to be greater than that of the master vehicle, and in the inner loop parameter update, the scene generation network performs high-frequency gradient optimization with a higher update step count than the master vehicle; after receiving the scene policy text output by the large language model, the scene generation network performs local parameter optimization in the continuous physical scene parameter vector space based on the high-frequency update and large step size mechanism, and dynamically iterates and adjusts the parameters according to the real-time update feedback of the master vehicle's capability profile, so that the adversarial difficulty of the generated scene is continuously maintained near the current capability boundary of the master vehicle.

8. The method for generating parameters for autonomous driving adversarial test scenarios based on profiles and large models according to claim 1, characterized in that: The method for constructing and calculating the direct preference optimization loss in step S4 is as follows: Based on the generation of the scene generation network and the objective feedback from the closed-loop physics engine, triples are automatically constructed, including prompt words representing the input, scene strategy text representing the winning preference, and scene strategy text representing the losing preference. If the physical scenario parameter vector generated by the scenario strategy text passes the difficulty validity check and causes the master vehicle to experience a high adversarial reward event in the physical simulation, then the text is marked as the scenario strategy text that wins the preference. If the scene policy text causes the generated parameters to violate the dynamic constraints and be intercepted by the hard constraint layer, or if the generated scene does not pose a threat to the master vehicle and falls into mode collapse, then the text is marked as a scene policy text that loses the preference; the parameters of the large language model are updated by maximizing the generation probability ratio of the scene policy text that wins the preference to the scene policy text that loses the preference.

9. The method for generating parameters for autonomous driving adversarial test scenarios based on profiles and large models according to claim 1, characterized in that: Before calculating the cross-modal graph-text consistency loss in step S4, an error backpropagation gating step is also included: obtaining the state prediction value of the currently generated physical scene parameter vector through the difficulty evaluation head in the scene generation network, and quantifying the execution deviation rate between the underlying parameter generation distribution and the high-level scene strategy text; when the execution deviation rate is greater than or equal to a preset threshold, truncating or attenuating the upward propagation of the cross-modal graph-text consistency loss gradient in the current round; when the execution deviation rate is less than the preset threshold, backpropagating the error gradient generated by the cross-modal graph-text consistency loss to the large language model in its entirety.

10. The method for generating parameters for autonomous driving adversarial test scenarios based on profiles and large models according to claim 1, characterized in that: The calculation method for cross-modal text-image consistency loss in step S4 is as follows: The system acquires the scene policy text output by the large language model inference engine, and the multi-view image sequence or bird's-eye view actually rendered by the underlying conditional world model based on the text; the system calls the pre-trained cross-modal model, and through the frozen text encoder and image encoder, maps the text and the rendered screen to the same high-dimensional feature space, respectively, to obtain text feature vectors and image feature vectors; the similarity is the cosine similarity between the text feature vector and the image feature vector, and the cross-modal image-text consistency loss is defined as 1 minus the cosine similarity; In step S4, when acquiring the scene image, a spatiotemporal dual-dimensional intelligent downsampling and viewpoint scheduling mechanism is adopted: In the spatial dimension, based on the spatial orientation of semantic entities in the scene strategy text, the sensor channel with the most representativeness is dynamically selected, or multimodal data is projected into a unified bird's-eye view semantic rendering image; In the temporal dimension, an event-driven sparse keyframe extraction strategy is adopted, and based on the trigger timestamp in the physical scene parameter vector or the frame-by-frame difficulty curve output by the difficulty assessment header, the moment when the adversarial difficulty peaks is automatically locked, and only the frame at that moment or the extremely short window frame is extracted as the scene image.

Images