Carrier-aware spatio-temporal layout attack method for automatic driving visual language model
Patent Information
- Application Number
- CN202611548141.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-10-08
- Publication Date
- 2026-11-03
AI Technical Summary
Existing safety testing methods for visual language models are difficult to effectively reflect the impact of forged driving text on the model's semantic reasoning in continuous dynamic driving videos. Furthermore, the injected text lacks a stable spatial association and semantic correspondence with the actual driving scenario, which can easily lead to incorrect scenario understanding and unsafe driving reasoning.
By acquiring continuous driving visual observation sequences and fixed task instructions, a proxy scene interpreter is used to analyze clean driving states, filter counterfactual target driving states and select compatible driving texts. Combined with risk-related visual carrier localization and local style matching, the text panel is rendered temporally smoothly. Finally, attack text is injected into the visual sequence and input into the model.
It achieves spatial association and temporal continuity between forged text and driving scenarios, enhances the semantic clarity of driving in attack text, is suitable for black-box model evaluation, reduces the model's scores in open-loop and closed-loop tasks, and affects closed-loop driving behavior.
Smart Images
Figure CN123031807A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of visual language model security testing technology, and in particular to a carrier perception spatiotemporal layout attack method for visual language models used in autonomous driving. Background Technology
[0002] With the development of autonomous driving technology, visual language models are increasingly being applied to tasks such as driving scene understanding, visual question answering, traffic element recognition, and driving semantic reasoning. These models can combine visual information such as vehicles, pedestrians, traffic lights, and road signs with natural language task instructions to comprehensively analyze the driving environment and driving behavior. Real-world driving scenarios also include textual information such as traffic signs, roadside signs, construction warning signs, lane indicators, and driver assistance interfaces. Visual language models can use text recognition and semantic understanding to incorporate this text as scene information for reasoning. To address the security of visual language models, existing research typically employs methods such as pixel perturbation, adversarial patches, visual cue injection, or text overlay to modify the visual input and observe changes in the model's output during scene understanding, visual question answering, and driving reasoning tasks.
[0003] Existing visual cue injection and typography attack methods typically focus on text content or text visibility in a single frame. The injected text lacks a stable spatial association with driving risk-related visual objects such as traffic lights, road signs, pedestrians, obstacles, and vehicles ahead. Furthermore, there may be a lack of clear semantic correspondence between the text content and actual driving conditions, traffic rules, and driving behavior. Simultaneously, some methods use fixed fonts, colors, or text boxes for overlay, failing to fully consider local visual features such as the surrounding background, borders, text size, and panel proportions, resulting in significant visual differences between the injected text and the original driving scene. For continuous driving videos, existing methods rarely consider the continuous changes in text content, position, and style between adjacent frames, easily leading to positional jumps, abrupt style changes, or content inconsistencies. Moreover, some attack methods rely on the internal parameters, gradients, or feature information of the tested model, limiting their applicability to black-box visual language models whose internal information is inaccessible. Therefore, they struggle to fully reflect the impact of scene text information on the model's driving semantic reasoning process under continuous dynamic driving observation. Summary of the Invention
[0004] To overcome the shortcomings of existing technologies, the purpose of this invention is to provide a carrier perception spatiotemporal layout attack method for autonomous driving visual language models, which solves the problem that autonomous driving visual language models are prone to misinterpreting forged driving text as credible visual evidence under continuous dynamic driving video observation, leading to incorrect scene understanding and unsafe driving reasoning.
[0005] To achieve the above objectives, the present invention provides the following solution: Attack methods targeting the carrier perception spatiotemporal layout of autonomous driving visual language models include: A continuous driving visual observation sequence, fixed task instructions, and a pre-constructed set of trusted driving text are acquired. The continuous driving visual observation sequence and fixed task instructions are analyzed using a proxy scene interpreter to obtain a clean driving state sequence. The clean driving state sequence includes: risk event type, scene state, driving action, and visual carrier type. The clean driving state sequence is subjected to counterfactual state transformation to obtain the counterfactual target driving state sequence. The compatible driving texts in the set of credible driving texts are filtered according to the counterfactual target driving state sequence and the visual carrier type to obtain the driving text to be injected. Based on the visual carrier type, the continuous driving visual observation sequence is used to locate risk-related visual carriers to obtain a set of candidate carrier regions. The candidate carrier region set is filtered based on carrier type compatibility, placement feasibility, and degree of occlusion to obtain the active carrier region; Based on the neighborhood visual attributes of the active carrier region, determine the local style parameters and the placement position of the text panel; The active carrier region is determined based on its effectiveness, visibility, and semantic compatibility with the counterfactual target driving state sequence; the driving text to be injected is consistent within the active carrier region. The text panel placement position and the local style parameters are temporally smoothed. The driving text to be injected is then rendered based on the smoothed text panel placement position and local style parameters to obtain the text panel canvas and text panel mask. Text is injected into the continuous driving visual observation sequence based on the text panel canvas and the text panel mask to obtain an adversarial driving visual sequence. The adversarial driving visual sequence and the fixed task instruction are input into the autonomous driving visual language model to obtain the model output results.
[0006] Preferably, a continuous driving visual observation sequence, fixed task instructions, and a pre-constructed set of trusted driving text are acquired. A proxy scene interpreter is then used to analyze the continuous driving visual observation sequence and fixed task instructions to obtain a clean driving state sequence, including: The agent scene interpreter is used to identify risk events, scene states, and driving actions in each frame of driving images to obtain risk event types, scene states, and driving actions. The risk event types include: traffic light events, pedestrian events, obstacle events, vehicles ahead events, and road sign events. The correlation between the risk event and the driver's behavior, as well as the corresponding visual carrier type, are determined based on the risk event type, the scenario state, and the driving action; the visual carrier type includes: traffic lights, traffic signs, roadside signs, road areas, obstacles, and vehicles.
[0007] Preferably, the clean driving state sequence is subjected to a counterfactual state transformation to obtain a counterfactual target driving state sequence. Compatible driving texts in the set of credible driving texts are then filtered based on the counterfactual target driving state sequence and the visual carrier type to obtain the driving text to be injected, including: Based on the risk event type, the scenario state, and the driving action, the clean driving state of each frame in the clean driving state sequence is converted into a counterfactual state to obtain a counterfactual target driving state sequence that conflicts with the safety meaning of the real scenario. Based on the counterfactual target driving state sequence, the candidate driving texts in the set of credible driving texts are semantically compatible to obtain semantically compatible driving texts; the candidate driving texts correspond to the target semantic state and the adaptation carrier type, respectively. Based on the visual carrier type, the semantically compatible driving text is filtered for carrier compatibility to obtain the compatible driving text; Select the driving text from the compatible driving text to obtain the driving text to be injected.
[0008] Preferably, risk-related visual carrier localization is performed on the continuous driving visual observation sequence according to the visual carrier type to obtain a candidate carrier region set, including: Generate carrier query conditions based on the visual carrier type; Based on the carrier query conditions, the carrier locator is used to locate the regions of each frame of driving images to obtain the candidate carrier region set; each candidate carrier region in the candidate carrier region set includes: candidate region bounding box, candidate region confidence score, and candidate region category label.
[0009] Preferably, the candidate carrier region set is filtered based on carrier type compatibility, placement feasibility, and degree of occlusion to obtain the active carrier region, including: The comprehensive score for each candidate carrier region is determined based on the candidate region confidence level, carrier type compatibility score, placement feasibility score, and occlusion penalty item. The candidate carrier regions are constrained and filtered based on the area ratio of the corresponding text panel in the driving image and the occlusion ratio between the corresponding text panel and the risk-related target region. Among the candidate carrier regions whose corresponding text panel area ratio is less than or equal to the preset maximum panel area ratio and whose occlusion ratio is less than or equal to the preset maximum occlusion ratio, the candidate carrier region with the highest comprehensive score is determined as the active carrier region; the selection expression for the active carrier region is: ;in, The active carrier region selected for frame t; This represents the i-th candidate region bounding box in frame t. Let be the set of candidate carrier regions for frame t; Let i be the confidence level of the i-th candidate region in the t-th frame; , , These are the weighting coefficients for compatibility score, placement feasibility score, and occlusion penalty item, respectively. Scoring the compatibility between candidate regions and carrier types; Scoring the feasibility of placing text panels near candidate areas; Penalties for excessively obscuring key risk targets.
[0010] Preferably, determining local style parameters and the placement position of the text panel based on the neighborhood visual attributes of the active carrier region includes: The local style parameters are obtained by extracting the neighborhood visual attributes of the active carrier region; the local style parameters include: background color, border color, text foreground color, border width, font size, and text panel aspect ratio. Determine candidate placement locations in the area surrounding the active carrier region; The text panel placement position is obtained by filtering the candidate placement positions that meet the preset conditions; the preset conditions include: the distance from the active carrier area meets the requirements, the alignment with the edge of the active carrier area, and the satisfaction of risk-related target occlusion constraints.
[0011] Preferably, determining the active carrier interval based on the effectiveness, visibility, and semantic compatibility with the counterfactual target driving state sequence of the active carrier region includes: Validity and visibility are determined for the active carrier region corresponding to each frame, and the validity and visibility results are obtained. Based on the counterfactual target driving state corresponding to each frame, the semantic compatibility of the active carrier region is judged to obtain the semantic compatibility judgment result. Based on the validity judgment result, the visibility judgment result, and the semantic compatibility judgment result, the range of frame indexes that continuously satisfy validity, visibility, and semantic compatibility is determined as the active carrier interval.
[0012] Preferably, the text panel placement position and the local style parameters are temporally smoothed, and the driving text to be injected is rendered based on the smoothed text panel placement position and local style parameters to obtain a text panel canvas and a text panel mask, including: The text panel placement position and local style parameters of the first frame of the active carrier interval are set to initial values for position smoothing and style smoothing, respectively. The text panel placement position in the current frame is smoothed according to the text panel placement position in the previous frame and the smoothed text panel placement position in the previous frame to obtain the smoothed text panel placement position in the current frame. Style temporal smoothing is performed based on the local style parameters of the current frame and the smoothed local style parameters of the previous frame to obtain the smoothed local style parameters of the current frame. Based on the smoothed text panel placement position and local style parameters, the consistent driving text to be injected within the active carrier range is typeset and rendered to obtain the text panel canvas and the text panel mask.
[0013] Preferably, text injection is performed on the continuous driving visual observation sequence based on the text panel canvas and the text panel mask to obtain an adversarial driving visual sequence, including: Set the text panel mask to a soft transparency mask that matches the size of the corresponding driving image; The text panel canvas is fused with the corresponding driving image within the active carrier region using the soft transparency mask to obtain an adversarial driving image; Set the driving image outside the active carrier range as the corresponding output frame; The adversarial driving images and the driving images outside the active vehicle range are arranged according to the frame order of the continuous driving visual observation sequence to obtain the adversarial driving visual sequence.
[0014] Preferably, the adversarial driving visual sequence and the fixed task instruction are input into the autonomous driving visual language model to obtain the model output, including: Under the condition that the fixed task instructions, the model parameters, inference interface and post-processing flow of the autonomous driving visual language model are consistent, the adversarial driving visual sequence and the fixed task instructions are input into the autonomous driving visual language model. The autonomous driving visual language model is used to perform scene understanding and driving semantic reasoning on the adversarial driving visual sequence to obtain the model output result; the model output result is one of the following: scene description result, traffic element identification result, driving question and answer result, and driving behavior prediction result.
[0015] The present invention discloses the following technical effects: This invention provides a carrier-based spatiotemporal layout attack method for autonomous driving visual language models. By constructing counterfactual driving semantics and filtering credible driving text, it addresses the problem of injected text lacking clear driving semantics and failing to form credible driving evidence, achieving the construction of attack text with clear driving meaning and target orientation. Through risk-related visual carrier localization, occlusion constraints, and local style matching layout, it solves the problems of insufficient spatial correlation between forged text and risk-related visual objects, as well as inconsistencies with the appearance of the driving scene, achieving positional and visual matching between forged text and the driving scene. By actively determining carrier intervals and preserving text content, positional smoothing, and style smoothing, it addresses the issues of flickering, positional jumps, and abrupt style changes in forged text within consecutive driving frames, achieving stable continuity of adversarial driving visual sequences in the temporal dimension. By injecting text only into driving visual observations and inputting the generated adversarial driving visual sequence into the autonomous driving visual language model, it solves the problem of existing attack methods' dependence on model internal parameters, gradients, and feature information, enabling the assessment of the security vulnerability of black-box autonomous driving visual language model scene text evidence. Attached Figure Description
[0016] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0017] Figure 1 A schematic diagram of the carrier perception spatiotemporal layout attack process for autonomous driving visual language models provided in an embodiment of the present invention; Figure 2 The image shows the result of the Dolphins model provided in the embodiment of the present invention; Figure 3 The result diagram of the DriveLM model provided in the embodiment of the present invention; Figure 4 This is a diagram of driving score (DS) results provided in an embodiment of the present invention; Figure 5 A route completion score (RC) result diagram provided in an embodiment of the present invention; Figure 6 This is a graph showing the violation score (IS) results provided in an embodiment of the present invention; Figure 7 The graph shows the transfer evaluation results of the general visual language model provided in the embodiments of the present invention. Detailed Implementation
[0018] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0019] The purpose of this invention is to provide a carrier perception spatiotemporal layout attack method for autonomous driving visual language models, which solves the problem that autonomous driving visual language models are prone to misinterpreting forged driving text as credible visual evidence under continuous dynamic driving video observation, leading to incorrect scene understanding and unsafe driving reasoning.
[0020] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.
[0021] Figure 1 This is a schematic diagram of the carrier perception spatiotemporal layout attack process for autonomous driving visual language models provided in an embodiment of the present invention, as shown below. Figure 1 As shown, this invention provides a carrier-based spatiotemporal layout attack method for autonomous driving visual language models, including: Step 100: Obtain the continuous driving visual observation sequence, fixed task instructions, and a pre-constructed set of trusted driving text. Analyze the continuous driving visual observation sequence and fixed task instructions using a proxy scene interpreter to obtain a clean driving state sequence. The clean driving state sequence includes: risk event type, scene state, driving action, and visual carrier type. Step 200: Perform counterfactual state transformation on the clean driving state sequence to obtain the counterfactual target driving state sequence. Based on the counterfactual target driving state sequence and the visual carrier type, filter the compatible driving texts in the set of credible driving texts to obtain the driving text to be injected. Step 300: Based on the visual carrier type, perform risk-related visual carrier localization on the continuous driving visual observation sequence to obtain a set of candidate carrier regions; Step 400: Filter the candidate carrier region set according to carrier type compatibility, placement feasibility, and degree of occlusion to obtain the active carrier region; Step 500: Determine the local style parameters and the placement position of the text panel based on the neighborhood visual attributes of the active carrier region; Step 600: Determine the active carrier interval based on the validity, visibility, and semantic compatibility with the counterfactual target driving state sequence of the active carrier region; the driving text to be injected is consistent within the active carrier interval; Step 700: Perform temporal smoothing on the text panel placement position and the local style parameters, and perform layout rendering on the driving text to be injected based on the smoothed text panel placement position and local style parameters to obtain the text panel canvas and text panel mask. Step 800: Inject text into the continuous driving visual observation sequence according to the text panel canvas and the text panel mask to obtain an adversarial driving visual sequence; Step 900: Input the adversarial driving visual sequence and the fixed task instruction into the autonomous driving visual language model to obtain the model output result.
[0022] Specifically, the implementation process of the carrier perception spatiotemporal layout attack method for autonomous driving visual language models proposed in this solution is as follows: Furthermore, problem definition and input processing: Specifically, this approach targets visual language models for autonomous driving scene understanding, driving question answering, and driving semantic reasoning tasks. Given a continuous sequence of driving visual observations and fixed task instructions, this approach modifies only the visual observation content, without altering the task instructions, model parameters, inference interface, or post-processing flow. Let the continuous driving visual observation sequence be... Fixed task instructions are The output of the autonomous driving visual language model is represented as: .
[0023] in, This represents the input driving visual sequence; Indicates the first Frame driving images; Indicates the video frame index; This indicates the total number of frames in the input sequence; Indicates a fixed task instruction; The visual language model representing the victimized autonomous driving system; Indicates model parameters; This indicates the model's output result; This represents the task-related output space. This output can be a scene description, traffic element recognition results, driving question-and-answer results, or driving behavior-related prediction results.
[0024] The adversarial visual sequence constructed in this scheme is represented as follows: .
[0025] in, This represents the adversarial driving visual sequence following the injection of typesetting text. This indicates the injection of transformation functions into the typed text; This indicates trusted driving text representing the scenario to be injected; This represents a sequence of text panel masks; This represents the rendered text panel canvas sequence.
[0026] For the The frame image, its adversarial image generation method is as follows: .
[0027] in, Indicates the first Frame-based adversarial images; Indicates the first A soft transparency mask for the frame text panel; Indicates the image height; Indicates the image width; The text panel indicates that the rendered canvas is the same size as the original image. This indicates an element-wise multiplication operation. This process only changes the visual input, not the task instructions. .
[0028] Preferably, the explanation of a clean driving state and the construction of a counterfactual objective: This approach first utilizes a proxy scene interpreter to analyze the original driving visual sequence and extract the clean driving state. This proxy scene interpreter is only used to construct the injected text and does not require access to the victim model's structure, parameters, gradients, or intermediate features. The clean driving state extraction process is represented as follows: .
[0029] in, This represents a sequence of clean driving states; This represents the proxy scenario interpreter; Parameters representing the proxy scenario interpreter; Indicates the first The clean driving state corresponding to the frame.
[0030] The clean driving state in each frame is represented as follows: .
[0031] in, Indicates the type of risk event, such as events related to traffic lights, pedestrians, obstacles, vehicles ahead, or road signs; It indicates the current observed scene state, such as red light, pedestrian presence, or obstacles blocking the road. This indicates the correlation between the risk event and the driver's behavior. This indicates driving actions consistent with the current real-world scenario, such as stopping, yielding, slowing down, or maintaining a stop. This indicates the type of visual carrier suitable for carrying the injected text, such as traffic lights, traffic signs, roadside signs, road areas, obstacles, or vehicles.
[0032] Subsequently, this scheme transforms the clean driving state into a counterfactual target driving state: .
[0033] in, Indicates the first The counterfactual target driving state of the frame; This represents a counterfactual state transition operator. This operator is used to convert the safety implications in a real-world scenario into opposite or conflicting target implications. For example, changing "stop at a red light" to "pass"; changing "yield when there are pedestrians" to "road clear"; and changing "slow down when there is an obstacle" to "road is safe".
[0034] Optionally, scene-based believable driving text generation: To avoid injecting arbitrary irrelevant text, this solution pre-constructs a set of trusted driving text. The text in this set is concise, driving-related, and can appear on traffic signs, roadside signs, or driving assistance prompts. The set of trusted driving text is represented as follows: .
[0035] in, Represents a set of trustworthy driving texts; Indicates the index of text candidate options; Indicates the total number of candidate texts; Indicates the first The target semantic state corresponding to each candidate text; Indicates the first The carrier type that the candidate text is adapted to; Indicates the first Candidate driving texts include, for example, SIGNAL GREEN, PROCEED, LANE CLEAR, SAFE TO GO, etc.
[0036] Given a counterfactual target driving state and carrier type The selected candidate text subset is then filtered to obtain the following: .
[0037] in, This represents a set of driving texts that are compatible with both the target state and the carrier type. This indicates that the semantics of the candidate text are compatible with the counterfactual target driving state; This indicates that the applicable carrier type of the candidate text is compatible with the current carrier type.
[0038] The final selected injection text is represented as follows: .
[0039] in, Indicates the first The target driving text selected in the frame; This indicates the module for generating or selecting driving text.
[0040] Furthermore, the positioning of risk-related carriers and the placement of text: This approach places the injected text near risk-related visual carriers, rather than at arbitrary image locations. For a given carrier type... The carrier positioning module generates candidate regions: .
[0041] in, Indicates the first The set of candidate carrier regions for a frame; Indicates a carrier locator; Indicates the parameters of the carrier locator; Indicates the carrier type Generated carrier query conditions.
[0042] The candidate region set is represented as: .
[0043] in, Indicates the first Frame number One candidate region bounding box; This indicates the confidence level of the candidate region; This indicates the category label for the candidate region; Indicates the first Number of candidate regions for a frame.
[0044] The selection criteria for active carriers are: .
[0045] in, Indicates the first The active carrier region selected in the frame; This indicates the candidate region that maximizes the score. This indicates the compatibility score between the candidate region and the carrier type; This indicates a feasibility score for placing text panels near the candidate area; This indicates penalties for excessively obscuring key risk targets; These represent the weighting coefficients for compatibility score, placement feasibility score, and occlusion penalty, respectively.
[0046] To prevent this solution from degenerating into a simple occlusion attack, constraints are imposed on the text panel area and the degree of occlusion: .
[0047] .
[0048] in, Indicates mask The sum of the absolute values of all elements in the array; This indicates the maximum proportion of the image area that the text panel is allowed to occupy; Indicates the first Risk-related target areas in the frame; Indicates the position of image pixels; This represents the total number of pixels within the risk-related target area; This indicates the percentage of obscuration of the risk target area by the text panel; This indicates the maximum allowable occlusion percentage.
[0049] Specifically, the style matches the rendering of the text panel: Selecting an active carrier Then, this scheme estimates local visual style parameters from the carrier's neighborhood: .
[0050] in, Indicates the first The set of local style parameters for a frame; This represents a style parameter estimation function; Indicates the background color; Indicates the border color; Indicates the foreground color of the text; Indicates the border width; Indicates font size; This indicates the aspect ratio of the text panel.
[0051] The position of the text panel is determined by the placement function: .
[0052] in, Indicates the first The placement of the frame text panel; This function represents the placement function. It prioritizes selecting a location that is close to the carrier, edge-aligned, and does not excessively obscure the key target.
[0053] Then, a text panel canvas and its corresponding mask are generated: .
[0054] .
[0055] in, Indicates style matching for the typography renderer; This represents the text panel mask generation function. Through this process, counterfactual driving semantics are rendered as text panels that match the appearance of the local scene.
[0056] Furthermore, temporally consistent propagation and adversarial sequence generation: For continuous driving observation, this scheme injects text only within the timeframe where the carrier is effective, visible, and semantically compatible with the target. The active carrier interval is defined as: .
[0057] in, Indicates the active carrier interval; Indicates existence; Indicates the first The active carrier of frame selection; Indicates the first The carrier type of the frame. For those that do not belong to... The frame does not perform text injection.
[0058] Within the active carrier region, this solution maintains consistency in text content: .
[0059] in, This indicates the driving text used uniformly within the active carrier range.
[0060] At the same time, the position of the text panel is smoothed over time: .
[0061] in, Indicates the first The position of the text panel after frame smoothing; Indicates the first The position of the text panel after frame smoothing; This represents the position smoothing coefficient.
[0062] Temporal smoothing of local style parameters: .
[0063] in, Indicates the first Style parameters after frame smoothing; Indicates the first Style parameters after frame smoothing; This represents the style smoothing coefficient.
[0064] For the first frame in the active carrier region Initialized as: .
[0065] in, This represents the first frame in the active carrier interval.
[0066] The final generated adversarial image is: .
[0067] in, Indicates the first The text panel mask generated under frame smoothing position and style is defined as follows: .
[0068] The final adversarial sequence is represented as: .
[0069] This adversarial sequence, along with the original task instructions, is input into the black-box autonomous driving visual language model: .
[0070] in, This represents the model's output under adversarial sequences. Through the above process, this scheme transforms counterfactual driving states into believable driving text for a specific scenario, and injects it into continuous driving visual sequences in a manner that is carrier-related, style-matched, and temporally consistent. This induces the autonomous driving visual language model to use the forged text as believable driving evidence in its reasoning.
[0071] Specifically, the results were verified as follows: To verify the effectiveness of the proposed carrier perception spatiotemporal layout attack method for autonomous driving visual language models, open-loop visual language model evaluation, general visual language model transfer evaluation, and closed-loop driving evaluation were conducted under a unified experimental setup. In the open-loop evaluation, the tested autonomous driving visual language models included Dolphins and DriveLM; in the general visual language model transfer evaluation, the tested models included GPT-5, Gemini-2.5-Pro, Qwen3-VL, and LLaVA-OneVision-1.5; and in the closed-loop driving evaluation, the tested model was LMDrive.
[0072] refer to Figures 2 to 7Experimental results show that this scheme can significantly reduce the scores of autonomous driving visual language models in driving question answering and scene reasoning tasks. On the Dolphins benchmark, this scheme reduced the FinalScore from 47.73 to 37.12, a relative decrease of 22.2%; on the DriveLM benchmark, this scheme reduced the FinalScore from 52.70 to 41.61, a relative decrease of 21.0%. In the general visual language model transfer evaluation, this scheme achieved the lowest scores on GPT-5, Gemini-2.5-Pro, Qwen3-VL, and LLaVA-OneVision-1.5, with an average relative decrease of 28.0%. In the LMDrive closed-loop driving evaluation, this scheme reduced the DrivingScore from 57.31 to 48.04, a relative decrease of 16.2%. These results indicate that the scene-based credible driving text generated by this scheme can not only affect open-loop driving semantic reasoning but also further influence closed-loop driving behavior.
[0073] Preferably, based on the above-described solution, the execution process of this embodiment consists of the following four steps: Step 1: Input sequence construction and black-box attack constraint initialization process.
[0074] The system acquires a continuous sequence of visual observations for autonomous driving. and fixed task instructions .in, Indicates the first Frame driving images, Indicates the total number of frames in the sequence. This indicates the task instructions received by the autonomous driving visual language model, such as scene understanding, driving question answering, or action reasoning instructions.
[0075] The original output of the autonomous driving visual language model is represented as follows: .
[0076] in, This represents the visual language model of the victimized autonomous driving system. Indicates model parameters, This represents the model's output. This approach is implemented under black-box conditions, without accessing the victim model's structure, parameters, gradients, or intermediate features; it only modifies the visual input sequence while maintaining the task instructions. Model parameters The reasoning process remains unchanged.
[0077] Step 2: Explanation of the clean driving state and construction of the counterfactual target state.
[0078] The system uses a proxy scene interpreter to extract clean driving states from the original driving visual sequence: .
[0079] in, This represents a sequence of clean driving states. This represents the proxy scenario interpreter. This represents the parameters of the proxy scenario interpreter.
[0080] The clean driving state of each frame can be represented as: .
[0081] in, Indicates the type of risk event. Indicates the current scene state. This indicates the correlation between the risk event and the vehicle. This represents reasonable driving actions in real-world scenarios. This indicates the type of visual carrier suitable for holding the injected text.
[0082] The system then switches from a clean driving state to a counterfactual target driving state: .
[0083] in, Indicates a counterfactual target driving state. This represents the counterfactual state transition operator. This step is used to convert real-world semantics into conflict semantics, such as changing "red light, stop" to "pass," or "pedestrians present, yield" to "road clear."
[0084] Step 3: Credible driving text generation and risk carrier perception rendering process.
[0085] The system is based on the counterfactual target driving state. and carrier type The injected text is selected from a pre-built set of trusted driving texts. The set of trusted driving texts is represented as follows: .
[0086] in, Indicates the first The target semantic state corresponding to each candidate text. This indicates the type of carrier the text is adapted to. Indicates candidate driving text, This indicates the total number of candidate texts.
[0087] The selected injection text is represented as: .
[0088] in, Indicates the first The selected driving text in the frame, This indicates the driving text selection module.
[0089] Subsequently, the system locates visual carriers related to the current risk event and selects the active carrier area: .
[0090] in, Indicates the first Set of candidate carrier regions for frames Indicates the first Candidate regions, Indicates the confidence level of the candidate region. This indicates the finally selected active carrier region. Indicates the carrier type compatibility score. Indicates the feasibility score for text placement. This indicates a penalty for occlusion. , , This represents the corresponding weighting coefficient.
[0091] The system estimates local style parameters such as background color, border color, text color, font size, border width, and panel ratio based on the neighborhood of the active carrier, and renders the selected text as a text panel that matches the style of the surrounding scene.
[0092] Step 4: Temporally consistent propagation and adversarial visual sequence generation process.
[0093] To ensure the stability and consistency of injected text in continuous driving videos, the system injects text within a time interval that is visible and valid to the active carrier and semantically compatible with the counterfactual target's driving state. This active interval is represented as: .
[0094] in, Indicates the active carrier region. This indicates that it exists. Indicates the first Frame active carrier, Indicates the first Frame carrier type.
[0095] Within the active region, the system maintains consistency in the injected text content and smoothly propagates the text panel position and visual style: .
[0096] .
[0097] in, Indicates the first Original placement position of the frame Indicates the smoothed placement position. Indicates the first Frame local style parameters, This represents the style parameters after smoothing. Indicates the position smoothing coefficient. This represents the style smoothing coefficient.
[0098] Finally, the first Frame adversarial images are represented as: .
[0099] in, Indicates the first Frame-based adversarial images, This represents the smoothed text panel mask. Indicates style matching typography renderer. This refers to the injected text used uniformly within the active interval. This indicates an element-wise multiplication operation.
[0100] The final generated adversarial visual sequence is: .
[0101] and compare it with the original mission instructions. When input into a black-box autonomous driving visual language model, the model may mistakenly take forged driving text as credible visual evidence, leading to incorrect scene understanding, incorrect driving reasoning, or an unsafe decision-making tendency.
[0102] The beneficial effects of this invention are as follows: (1) The construction of counterfactual driving semantics combined with the generation of credible driving text makes the injected content have clear driving semantics and target orientation, which strengthens the ability of fake text to interfere with the model's understanding of the scene and the driving reasoning process.
[0103] (2) Risk-related carrier positioning anchors the forged text near visual objects related to the current driving risk, and combines local style matching rendering to make the text more closely resemble real traffic prompts or driving assistance prompts in terms of spatial location and visual appearance, thereby enhancing the credibility of the scene.
[0104] (3) To address the issues of text flickering, position jumps, and style abrupt changes that are prone to occur in continuous driving videos, a text content preservation and position and style smoothing mechanism is adopted to ensure that the injected text can maintain good continuity and stability between adjacent frames.
[0105] (4) This invention only modifies the driving vision observation content, while keeping the task instructions, model parameters and reasoning process consistent. It does not require obtaining the gradient and internal features of the victim model, so it can be applied to the safety assessment of black box autonomous driving vision language models.
[0106] (5) Experimental results show that the present invention can not only reduce the evaluation scores of autonomous driving visual language models and general visual language models in open-loop tasks, but its influence can also be extended to closed-loop driving behavior, thus demonstrating cross-model transfer ability and influence on the closed-loop driving process.
[0107] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other.
[0108] This document uses specific examples to illustrate the principles and implementation methods of the present invention. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of the present invention. Furthermore, those skilled in the art will recognize that, based on the ideas of the present invention, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of the present invention.
Claims
1. A carrier-based spatiotemporal layout attack method for visual language models in autonomous driving, characterized in that, include: The observation sequence, task instructions, and trusted driving text set are obtained. The observation sequence and task instructions are analyzed using a proxy scene interpreter to obtain a clean driving state sequence including visual carrier types. The clean driving state sequence is subjected to counterfactual state transformation to obtain a counterfactual state sequence. The compatible driving texts in the set of credible driving texts are filtered according to the counterfactual state sequence and the visual carrier type to obtain the driving text to be injected. Based on the visual carrier type, the observation sequence is risk-related visual carrier localization is performed to obtain a set of candidate carrier regions; The candidate carrier region set is filtered based on carrier type compatibility, placement feasibility, and degree of occlusion to obtain the active carrier region; The local style parameters and the placement of the text panel are determined based on the neighborhood visual attributes of the active carrier region. The active carrier interval is determined based on the effectiveness, visibility, and semantic compatibility with the counterfactual state sequence of the active carrier region. The text to be injected driving text is typed and rendered using the smoothed text panel placement position and local style parameters to obtain the text panel canvas and text panel mask. Text injection is performed on the observation sequence based on the text panel canvas and the text panel mask to obtain an adversarial driving vision sequence. The adversarial driving visual sequence and the task instructions are input into the autonomous driving visual language model to obtain the model output results.
2. The carrier perception spatiotemporal layout attack method for autonomous driving visual language models according to claim 1, characterized in that, The system acquires observation sequences, task instructions, and a set of trusted driving text. It then analyzes the observation sequences and task instructions using a proxy scene interpreter to obtain a clean driving state sequence, including visual carrier types: The agent scene interpreter is used to identify risk events, scene states, and driving actions in each frame of driving images to obtain risk event types, scene states, and driving actions. The risk event types include: traffic light events, pedestrian events, obstacle events, vehicles ahead events, and road sign events. The correlation between the risk event and the driver's behavior, as well as the corresponding visual carrier type, are determined based on the risk event type, the scenario state, and the driving action; the visual carrier type includes: traffic lights, traffic signs, roadside signs, road areas, obstacles, and vehicles.
3. The carrier perception spatiotemporal layout attack method for autonomous driving visual language models according to claim 1, characterized in that, The clean driving state sequence is subjected to a counterfactual state transformation to obtain a counterfactual state sequence. Compatible driving texts in the trusted driving text set are then filtered based on the counterfactual state sequence and the visual carrier type to obtain the driving text to be injected, including: Based on the risk event type, scenario state, and driving action, the clean driving state of each frame in the clean driving state sequence is converted into a counterfactual state sequence, resulting in a counterfactual state sequence that conflicts with the safety implications of the real scenario. Based on the counterfactual state sequence, the candidate driving texts in the set of credible driving texts are semantically compatible to obtain semantically compatible driving texts; the candidate driving texts correspond to the target semantic state and the adapted carrier type, respectively. Based on the visual carrier type, the semantically compatible driving text is filtered for carrier compatibility to obtain the compatible driving text; Select the driving text from the compatible driving text to obtain the driving text to be injected.
4. The carrier perception spatiotemporal layout attack method for autonomous driving visual language models according to claim 1, characterized in that, Based on the visual carrier type, the observation sequence is risk-related visual carrier localization is performed to obtain a candidate carrier region set, including: Generate carrier query conditions based on the visual carrier type; Based on the carrier query conditions, the carrier locator is used to locate the regions of each frame of driving images to obtain the candidate carrier region set; each candidate carrier region in the candidate carrier region set includes: candidate region bounding box, candidate region confidence score, and candidate region category label.
5. The carrier perception spatiotemporal layout attack method for autonomous driving visual language models according to claim 1, characterized in that, The candidate carrier region set is filtered based on carrier type compatibility, placement feasibility, and degree of occlusion to obtain the active carrier region, including: The comprehensive score for each candidate carrier region is determined based on the candidate region confidence level, carrier type compatibility score, placement feasibility score, and occlusion penalty item. The candidate carrier regions are constrained and filtered based on the area ratio of the corresponding text panel in the driving image and the occlusion ratio between the corresponding text panel and the risk-related target region. Among the candidate carrier regions whose corresponding text panel area ratio is less than or equal to the preset maximum panel area ratio and whose occlusion ratio is less than or equal to the preset maximum occlusion ratio, the candidate carrier region with the highest comprehensive score is determined as the active carrier region; the selection expression for the active carrier region is: ;in, The active carrier region selected for frame t; This represents the i-th candidate region bounding box in frame t. Let be the set of candidate carrier regions for frame t; Let i be the confidence level of the i-th candidate region in the t-th frame; , , These are the weighting coefficients for compatibility score, placement feasibility score, and occlusion penalty item, respectively. Scoring the compatibility between candidate regions and carrier types; Scoring the feasibility of placing text panels near candidate areas; Penalties for excessively obscuring key risk targets.
6. The carrier perception spatiotemporal layout attack method for autonomous driving visual language models according to claim 1, characterized in that, Determining local style parameters and text panel placement based on the neighborhood visual attributes of the active carrier region includes: The local style parameters are obtained by extracting the neighborhood visual attributes of the active carrier region; the local style parameters include: background color, border color, text foreground color, border width, font size, and text panel aspect ratio. Determine candidate placement locations in the area surrounding the active carrier region; The text panel placement position is obtained by filtering the candidate placement positions that meet the preset conditions; the preset conditions include: the distance from the active carrier area meets the requirements, the alignment with the edge of the active carrier area, and the satisfaction of risk-related target occlusion constraints.
7. The carrier perception spatiotemporal layout attack method for autonomous driving visual language models according to claim 1, characterized in that, Determining the active carrier interval based on the validity, visibility, and semantic compatibility with the counterfactual state sequence of the active carrier region includes: Validity and visibility are determined for the active carrier region corresponding to each frame, and the validity and visibility results are obtained. Based on the counterfactual target driving state corresponding to each frame, the semantic compatibility of the active carrier region is judged to obtain the semantic compatibility judgment result. Based on the validity judgment result, the visibility judgment result, and the semantic compatibility judgment result, the range of frame indexes that continuously satisfy validity, visibility, and semantic compatibility is determined as the active carrier interval.
8. The carrier perception spatiotemporal layout attack method for autonomous driving visual language models according to claim 1, characterized in that, The text to be injected is rendered using the smoothed text panel placement position and local style parameters to obtain a text panel canvas and a text panel mask, including: The text panel placement position and local style parameters of the first frame of the active carrier interval are set to initial values for position smoothing and style smoothing, respectively. The text panel placement position in the current frame is smoothed according to the text panel placement position in the previous frame and the smoothed text panel placement position in the previous frame to obtain the smoothed text panel placement position in the current frame. Style temporal smoothing is performed based on the local style parameters of the current frame and the smoothed local style parameters of the previous frame to obtain the smoothed local style parameters of the current frame. Based on the smoothed text panel placement position and local style parameters, the consistent driving text to be injected within the active carrier range is typeset and rendered to obtain the text panel canvas and the text panel mask.
9. The carrier perception spatiotemporal layout attack method for autonomous driving visual language models according to claim 1, characterized in that, Text injection is performed on the observation sequence based on the text panel canvas and the text panel mask to obtain an adversarial driving visual sequence, including: Set the text panel mask to a soft transparency mask that matches the size of the corresponding driving image; The text panel canvas is fused with the corresponding driving image within the active carrier region using the soft transparency mask to obtain an adversarial driving image; Set the driving image outside the active carrier range as the corresponding output frame; The adversarial driving images and the driving images outside the active vehicle range are arranged according to the frame order of the observation sequence to obtain the adversarial driving visual sequence.
10. The carrier perception spatiotemporal layout attack method for autonomous driving visual language models according to claim 1, characterized in that, The adversarial driving visual sequence and the task instructions are input into the autonomous driving visual language model to obtain the model output results, including: Under the condition that the task instructions, the model parameters, inference interface and post-processing flow of the autonomous driving visual language model are consistent, the adversarial driving visual sequence and the task instructions are input into the autonomous driving visual language model. The autonomous driving visual language model is used to perform scene understanding and driving semantic reasoning on the adversarial driving visual sequence to obtain the model output result; the model output result is one of the following: scene description result, traffic element identification result, driving question and answer result, and driving behavior prediction result.