A Security Defense Method for Jailbreak-Related Representation Shift in a Visual Language Model
By calculating the jailbreak direction vector to correct the hidden state representation of the visual language model, the problem of inaccurate separation of causal components in jailbreak attacks is solved, achieving efficient defense and lossless security improvement for benign missions.
Patent Information
- Application Number
- CN202611612166.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-10-19
- Publication Date
- 2026-11-17
AI Technical Summary
Existing visual language models are unable to accurately separate jailbreak-related causal components introduced by images when facing jailbreak attacks, resulting in insufficient defense or loss of general capabilities.
By collecting multimodal harmful inputs, the hidden state representation of the frozen visual language model is extracted, the centroids of the jailbreak and refusal samples are calculated, the jailbreak direction vector is obtained, and the hidden state representation is corrected during inference, the jailbreak-related offset components are removed, and normal semantic information is preserved.
It achieves a 72% absolute reduction in jailbreak attacks, significantly improving security, while having almost no impact on benign tasks, maintaining the model's generality, and possessing cross-scenario generalization ability and low computational overhead.
Smart Images

Figure CN123116293A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of security technology for multimodal large models of artificial intelligence, and in particular to a security defense method for jailbreak-related representation offset of a visual language model. Background Technology
[0002] Visual language models (VLMs) have demonstrated exceptional capabilities in tasks such as open-world visual understanding and complex reasoning by integrating a visual encoder with a large language model skeleton. However, the security alignment of VLMs with their underlying large language model skeletons degrades significantly after integrating the visual modality. For example, simply appending a blank image to a malicious text prompt can dramatically increase the jailbreak success rate of VLMs on malicious datasets, indicating that the introduction of the visual modality introduces a systemic security vulnerability.
[0003] Existing research has proposed the security perception failure hypothesis to explain the phenomenon of escapes using visual language models. This hypothesis posits that in a plain text setting, visual language models can correctly identify and reject harmful input. However, the addition of visual input impairs the model's security perception ability, making it unable to distinguish between harmful and benign input, thus leading to escapes. However, this hypothesis primarily relies on implicitly harmful data, where textual cues are rewritten in a harmless form, and harmful intent is conveyed solely through images. This type of data introduces two types of ambiguity: first, because the text is rewritten, observed escape behavior may stem from the rewritten wording rather than the image, causing causal ambiguity; second, while concrete harmful concepts (such as bombs and violence) can be inferred from images, abstract harmful concepts (such as fraud, privacy violations, and economic harm) are difficult to infer solely from images, causing semantic ambiguity. These two types of ambiguity limit the applicability of the security perception failure hypothesis.
[0004] Inference-time defense methods against jailbreak attacks on visual language models are mainly divided into two categories: (1) Input layer defense methods, including security cue injection (such as AdaShield, which guides the model to check for harmful content in images and text by adding security instructions to the input), and converting visual input into text descriptions (such as ECSO, which activates the security mechanism built into the skeleton of the large language model by converting images into text descriptions). These methods often impair the general capabilities of the model or introduce significant computational overhead; (2) Representation layer defense methods, including pulling multimodal representations back to the text domain (such as ShiftDC, which identifies the security direction by calculating the difference between the average representations of harmful and benign plain text inputs and removing security-related offset components in the multimodal representations), extracting the offset vector introduced by the visual input and subtracting it from the multimodal hidden state (such as CMRM, which calculates the offset vector at the sample level and aligns the multimodal representation to the original large language model state by comparing the hidden state of a plain text query with the hidden state of the same query plus a blank image). However, existing representation layer methods have failed to accurately separate the specific components causally related to the jailbreak behavior from the total representation offset introduced by the image, which may lead to insufficient defense or loss of useful image semantic information, resulting in a loss of the model's general capabilities.
[0005] Therefore, there is an urgent need for a reasoning-time defense method that can accurately locate and eliminate jailbreak-related components in the representation space while preserving normal visual semantic information. Summary of the Invention
[0006] The purpose of this invention is to provide a security defense method for a visual language model based on jailbreak-related representation offset, in order to solve the technical problem in the prior art that jailbreak-related causal components cannot be accurately separated from the total representation offset introduced from the image, resulting in insufficient defense or loss of general capabilities.
[0007] To achieve the above objectives, the present invention adopts the following technical solution: A security defense method for a visual language model based on jailbreak-related representation shift includes the following steps: collecting several multimodal malicious inputs; inputting the multimodal malicious inputs into a frozen visual language model to generate responses; judging each response as jailbreak or rejection based on preset evaluation rules; sampling several jailbreak samples and several rejection samples respectively; inputting the jailbreak samples and rejection samples into the frozen visual language model respectively; extracting the hidden state representation of the last word position in each layer of a preset layer set; averaging the hidden state representations of the jailbreak samples and rejection samples to obtain the jailbreak centroid and rejection centroid respectively; calculating the difference between the two and performing L2 normalization to obtain the jailbreak direction vector, which remains fixed in subsequent inference; for the multimodal input to be inferred, inputting the multimodal input into the visual language model without enabling intervention; extracting the hidden state representation of the last word position in each layer of the preset layer set; constructing a plain text input corresponding to the multimodal input. Without intervention, the plain text input is fed into the visual language model, and the hidden state representations of the last word positions at the same layer are extracted. The difference between the hidden state representations of the multimodal input and the plain text input at each layer is calculated to obtain the total representation offset introduced by the image. The total representation offset introduced by the image is projected onto the jailbreak direction vector to obtain a jailbreak-related offset scalar value. The jailbreak-related offset scalar value is divided by the L2 norm of the total representation offset to obtain the normalized jailbreak-related offset. At the first decoding step, it is determined whether the normalized jailbreak-related offset exceeds a preset threshold. An intervention-enabled forward propagation is performed on the original multimodal input. If the normalized jailbreak-related offset exceeds the preset threshold, the product of the jailbreak-related offset scalar value and the jailbreak direction vector is subtracted from the last word hidden state representation of the corresponding layer in this forward propagation to obtain the corrected hidden state representation. Based on the corrected hidden state representation, autoregressive generation continues, and a safe response is output.
[0008] Furthermore, in step S1, the preset evaluation rule adopts a consistency screening strategy of three methods: keyword matching, evaluation by the first language model, and evaluation by the second language model. Only samples with completely consistent evaluation results from the three methods are retained, while samples with conflicting evaluation results are discarded. The number of samples for both the jailbreak samples and the refusal to answer samples is 50.
[0009] Furthermore, in step S1, the sample selection for estimating the jailbreak direction vector adopts a conflict-discarding strategy, retaining only samples whose judgment results are completely consistent across the three methods of keyword matching, first language model evaluation, and second language model evaluation, in order to reduce annotation noise; when evaluating defense performance, a majority voting strategy is used to determine the response, and the response is labeled as jailbreak only when at least two methods consistently determine it as jailbreak.
[0010] Furthermore, in step S1, both the jailbreak samples and the rejection samples come from the same explicit malicious dataset, where the text prompts explicitly contain malicious intent; the jailbreak direction vector, estimated from the explicit malicious dataset, can be directly generalized to implicit malicious scenarios and adversarial attack scenarios.
[0011] Furthermore, the formula for calculating the jailbreak direction vector in step S2 is as follows: ,in Let the vector be the difference between the centroids of the escape and the centroids of the refusal to answer. and They represent the first The centroids of the hidden states of the escaped and rejected samples at each level. For layer index, For the first The escape direction vector of the layer, The L2 norm is represented; the escape direction vector is a unit vector pointing from the rejection centroid to the escape centroid, representing the internal transition direction from the rejection state to the escape state in the model representation space.
[0012] Furthermore, in step S4, the plain text input is constructed by removing the image and corresponding visual input, retaining only the text prompt, and processing it according to the model's plain text input method; the formula for calculating the total representation offset is... ,in For the first The total representation offset vector introduced by the layer image. For layer index, For multimodal input The hidden state representation of the last word element, Represents an image. This indicates a text prompt. For corresponding plain text input The hidden state representation of the last word element, This indicates that it does not contain images and corresponding visual input.
[0013] Furthermore, the calculation formula for the jailbreak-related offset scalar value in step S5 is as follows: ,in For the first The jailbreak-related offset scalar value of the layer. Indicates transpose. The total representation offset vector introduced for the image. For the first The escape direction vector of the layer. For layer index; the calculation formula for the normalized jailbreak-related offset in step S6 is as follows: ,in For the first The normalized jailbreak-related offset of the layer, The L2 norm is represented, and the normalized jailbreak-related offset ranges from [-1, 1]. The normalized jailbreak-related offset of jailbreak samples is usually greater than 0.3, while the normalized jailbreak-related offset of benign samples is usually less than 0.1.
[0014] Furthermore, the calculation formula for the correction in step S7 is as follows: ,when ,in For the revised first Hidden state representation of end-layer lexical units, For the generation of the first step before correction in the forward propagation with intervention. Hidden state representation of end-layer lexical units, i.e. For multimodal input The hidden state representation of the last word element, For the first The offset scalar value related to the layer jailbreak. For the first The escape direction vector of the layer. For layer index; For the first The normalized jailbreak-related offset of the layer, when The correction is performed in time, where The preset threshold is preferably in the range of 0.1 to 0.3, with the optimal value being 0.2. The correction operation only removes the component along the jailbreak direction in the hidden state representation of the last word, while retaining the semantic information component unrelated to jailbreak. The correction subtracts the projection component of the total representation offset introduced by the image in the jailbreak direction from the multimodal hidden state.
[0015] Furthermore, the preset layer set covers all decoding layers of the visual language model; steps S5 and S6 are calculated on each preset layer respectively, and step S7 is executed according to the layer order of the model forward propagation, calculating the jailbreak-related offset scalar value and normalized jailbreak-related offset of each layer respectively, and correcting the hidden state representation of the layer when the threshold condition is met.
[0016] Furthermore, the calculation of the total representation offset introduced by the image in steps S3 and S4 is completed before the first decoding step of the inference, requiring two uninterrupted reference forward propagations; then, an interrupted generative forward propagation is performed on the original multimodal input, and autoregressive generation continues; in step S7, the representation correction is only performed once in the pre-filling stage before the generation of the first output lexical, and the correction is not applied again in subsequent decoding.
[0017] The technical solution of this invention is based on the following key findings: By performing principal component analysis and linear probing analysis on the distribution of benign samples, refusal samples, and escape samples in the representation space of the visual language model, it was found that the three types of samples are highly separable in the intermediate and deep representation spaces, with escape samples occupying an independent region in the representation space. Simultaneously, a large number of escape responses contain suggestive statements regarding risks, illegality, or ethical issues, indicating that the model can still recognize harmful intentions when an escape occurs. These findings suggest that the escape of the visual language model does not stem from an inability to recognize harmful intentions, but rather from the model recognizing harmful intentions but failing to trigger a refusal behavior, i.e., entering a unique internal state of "recognition but failure to refuse." The addition of an image causes a shift in the representation space, pushing the sample from the refusal state to the escape state.
[0018] Preferably, the jailbreak direction vector is estimated only once on the frozen visual language model, and a highly consistent estimation result can be obtained using 50 jailbreak samples and 50 rejection samples.
[0019] Preferably, the representation correction subtracts the projection component of the total representation offset introduced by the image in the jailbreak direction from the multimodal hidden state, so that the generality of benign input is essentially unaffected.
[0020] Preferably, the preset threshold is 0.2, which achieves the optimal balance between security and versatility.
[0021] Compared with existing technologies, this invention has the following beneficial effects: 1. It proposes the "identification but failure to refuse" hypothesis and the corresponding jailbreak direction definition and jailbreak-related offset quantification method. For the first time, it accurately separates the causally related components of jailbreak behavior from the total representation offset introduced by the image, providing a unified quantitative explanation for jailbreak behavior in visual language models. It covers multiple jailbreak scenarios such as explicit malicious, implicit malicious, and adversarial attacks, overcoming the limitations of existing security perception failure hypotheses, which are only applicable to implicit malicious data and have causal and semantic ambiguities. 2. Based on the quantitative analysis of jailbreak-related offsets, it proposes a defense method during inference. By removing only jailbreak-related offset components, it restores the model's ability to refuse responses. On multiple explicit malicious datasets such as HADES, MM-SafetyBench, and RedTeam2K, it achieves an absolute decrease in attack success rate of up to approximately 72 percentage points. It also achieves significant defense effects in implicit malicious and adversarial attack scenarios. Compared with existing methods such as AdaShield, ECSO, ShiftDC, and CMRM, it achieves the best or second-best results on all evaluation lattices. 3. The correction subtracts the projection component of the total representation offset introduced by the image onto the jailbreak direction. Experiments show that it has almost no impact on benign tasks (MM-Vet, ScienceQA, MME), and its generality is almost zero on the three visual-language models LLaVA-1.5-7B, ShareGPT4V-7B, and InternVL-Chat-19B, maintaining the model's generality intact. 4. The jailbreak direction can be estimated from only 50 jailbreak and refusal samples from each of the explicit harmful datasets. This direction shows high consistency across different datasets and image types, with cross-dataset cosine similarity exceeding 0.7, indicating that the jailbreak direction is an inherent structural feature of the model rather than a product of a specific data distribution, demonstrating strong cross-scene generalization ability. 5. The entire defense process requires no training or additional models, only two pre-filled forward propagations before the first decoding step. The computational cost is negligible compared to subsequent autoregressive generation, making it suitable for real-time deployment. Attached Figure Description
[0022] Figure 1a This is a schematic diagram of the overall framework of the visual language model security defense method of the present invention.
[0023] Figure 1b This is a schematic diagram illustrating the algorithm principle of the visual language model security defense method of the present invention.
[0024] Figure 2 This diagram illustrates the principal component analysis, distance analysis, and linear probe analysis of three types of samples—benign, non-responding, and escaped—in the visual language model representation space.
[0025] Figure 3 This is a schematic diagram illustrating the quantitative analysis of jailbreak-related offsets under different jailbreak scenarios.
[0026] Figure 4 This is a schematic diagram illustrating the impact of a preset threshold τ on security and versatility.
[0027] Figure 5 This is a schematic diagram showing the effect of comparing the success rate of attacks in various jailbreak attack scenarios with the baseline method. Detailed Implementation
[0028] Figure 1a , Figure 1b , Figure 2 , Figure 3 , Figure 4 , Figure 5 The six attached figures constitute a complete and rigorous logic of "proposing a hypothesis → defining indicators → verifying indicators → parameter tuning → final result".
[0029] Figure 1a , Figure 1b (Overall framework diagram): Shows the complete technical solution architecture of the offline stage (jailbreak direction estimation) and the online stage (offset calculation, threshold judgment, representation correction).
[0030] The present application will now be described in further detail with reference to the accompanying drawings and embodiments. It should be understood that the following embodiments are used to illustrate the technical solutions of the present application, and are not intended to limit the scope of protection of the present application.
[0031] Example 1 like Figure 1a , Figure 1b As shown, the visual language model security defense framework based on jailbreak-related representation offset proposed in this application consists of two stages: offline jailbreak direction estimation and online jailbreak-related offset quantization and representation correction. The offline stage only needs to be executed once to obtain the jailbreak direction vectors of each preset layer and store them in a fixed manner; the online stage calculates the jailbreak-related offset for each input sample and performs representation correction when a threshold condition is met.
[0032] The technical solution of this application is based on the following key findings: Figure 2As shown, principal component analysis (PCA) and linear probe analysis were performed on the distribution of benign samples, refusal samples, and escape samples in the visual language model representation space. The results revealed that the three types of samples are highly separable in the intermediate and deep representation spaces. The first principal component (PC1) distinguishes benign inputs from all harmful inputs, and the second principal component (PC2) further distinguishes refusal behavior from escape behavior. Escape samples occupy an independent region in the representation space. Furthermore, a large number of escape responses contain suggestive statements regarding risks, illegality, or ethical issues, indicating that the model can still recognize harmful intent in the input even when an escape occurs. These findings suggest that the escape of the visual language model does not stem from an inability to recognize harmful intent, but rather from the model recognizing harmful intent but failing to trigger refusal behavior, i.e., entering a unique internal state of "recognition but failure to refuse." The addition of the image causes a shift in the representation space, pushing the sample from the refusal state to the escape state.
[0033] In one implementation, a complete implementation process of a security defense method based on the LLaVA-1.5-7B visual language model is provided, combining... Figure 1a , Figure 1b Please provide an explanation.
[0034] I. System Configuration. The hardware configuration used in this implementation is as follows: Operating system Ubuntu 22.04, processor Intel(R) Xeon(R) series CPU, GPU NVIDIA A100 80GB×1. The software configuration is as follows: Visual language model LLaVA-1.5-7B (based on the Vicuna-7B language model skeleton, with 32 decoding layers, 4096 hidden dimensions, and a CLIP ViT-L / 14 visual encoder), inference framework PyTorch, inference precision float16, greedy decoding for generation (do_sample=False), maximum generation length of 128 tokens, random seed fixed at 0, and batch size of 1. All visual language models are loaded from the official Hugging Face model library. Input is processed using the apply_chat_template method of the model-specific processor, applying the dialogue template and returning a PyTorch tensor.
[0035] II. Constructing the escape sample set and the refusal response sample set, and estimating the escape direction (steps S1–S2, offline stage). This implementation uses the explicit harmful subset from the HADES dataset. The HADES dataset contains 750 explicit harmful text prompts, covering five categories: animal cruelty, financial crime, privacy violation, self-harm, and violence. For each harmful text prompt, HADES provides three types of images: SD (images related to the text prompt generated by Stable Diffusion), TYPO (formatted images of harmful keywords), and SD+TYPO (a combination of the two), forming a total of 2250 multimodal samples.
[0036] All 2250 samples were input into the frozen LLaVA-1.5-7B to generate responses. A conflict-discarding strategy was used to filter the samples: keyword matching, Qwen3Guard-Gen-8B, and Llama-Guard4-12B methods were used to judge each response, and only samples with completely consistent results from all three methods were retained to reduce annotation noise and ensure the accuracy of the jailbreak direction vector estimation. Fifty jailbreak samples and 50 rejection samples were randomly sampled from the filtered jailbreak samples and jailbreak centroids were calculated on all 32 decoding layers. (No. The mean vector of the hidden state representation of the layered jailbreak sample and the centroid of the rejection response. (No. The mean vector of the hidden state representation of the layered rejected sample, according to the formula Calculate the escape direction vector ;in For layer index (value range 0 to 31). For the first The unit vector for the escape direction of the layer. Let the vector be the difference between the centroids of the escape and the centroids of the refusal to answer. Describing the L2 norm, This represents a 4096-dimensional real vector space (corresponding to the hidden dimension of LLaVA-1.5-7B). A total of 32 escape direction vectors were obtained, which were stored in a fixed location for reuse during the online inference stage. The estimation of the escape direction vectors exhibits high sample efficiency: in 10 repeated random sampling trials on LLaVA-1.5-7B, the cosine similarity between the escape direction vectors estimated using 50 escape samples and 50 rejection samples consistently remained highly consistent with the directions estimated using the full dataset, indicating that a small number of samples are sufficient to reliably estimate the escape direction.
[0037] III. Online Reasoning and Representation Correction (Steps S3–S7). For the multimodal input to be reasoned... (in Represents an image. (Indicating text hints), two uninterrupted reference forward propagations are performed before the first decoding step: the first is a multimodal forward propagation with images, extracting the hidden state representation of the last word position at each layer. (No. Multimodal input on layer The first is the hidden state vector of the last word; the second is the plain text forward propagation without images (removing images and corresponding visual inputs, retaining only text prompts). And process it according to the model's plain text input method, constructing plain text input. ,in (This indicates that the hidden state representation does not include images and corresponding visual input), extracting the hidden state representation at the end-word position of the same layer. (No. Plain text input on layer (The last word hidden state vector). Calculate the total representation offset introduced by the image. (That is, multimodal input and plain text input in the first) The difference vector between the hidden states of the last word in the layer is projected onto the jailbreak direction vector to obtain the jailbreak-related offset scalar value. (in Indicates transpose. (The value is a scalar reflecting the projection of the total representation offset onto the escape direction). Dividing this by the L2 norm of the total representation offset yields the normalized escape-related offset. ( The value ranges from [-1, 1], representing the normalized proportion of the total offset aligned with the jailbreak direction. For example... Figure 3 As shown, in explicit malicious, implicit malicious, and adversarial attack scenarios, the normalized jailbreak correlation bias of jailbreak samples in the mid-to-deep layers is significantly higher than that of rejected samples (usually greater than 0.3); while in benign data scenarios, the bias values of various benign baseline data are close to 0 (usually less than 0.1). A preset threshold τ=0.2 is used, and judgment is made layer by layer across all 32 layers. Then, intervention-based forward propagation is performed on the original multimodal input: for those satisfying... The layer performs the correction operation. (in For the revised first The hidden state representation of the last word in the layer is calculated by subtracting the projection component of the total representation offset introduced by the image in the escape direction from the multimodal hidden state. The original representation in the correction formula refers to the current representation of the corresponding layer in the forward propagation of this generation, and the projection scalar is calculated by the aforementioned reference forward propagation. The correction is only performed once in the pre-filling stage before generating the first output word, and no additional calculation is required in the subsequent autoregressive generation process. The computational cost is negligible compared to the autoregressive generation. Steps S5 and S6 are calculated separately on each preset layer, and step S7 is performed according to the layer order of the model forward propagation: the escape-related offset scalar value of each layer is calculated independently. Normalized jailbreak-related offsets and in satisfying At that time, a correction operation is performed on the hidden state representation of that layer.
[0038] IV. Verification of Defense Effectiveness in Explicitly Harmful Scenarios. This implementation evaluates the defense effectiveness on three explicitly harmful datasets: HADES, MM-SafetyBench, and RedTeam2K. Each text prompt is paired with one of three types of images: SD, TYPO, and SD+TYPO. The defense performance is evaluated using a majority voting strategy: keyword matching, Qwen3Guard-Gen-8B, and Llama-Guard4-12B methods are used. The response is labeled as jailbroken only when at least two methods consistently determine that it is jailbroken. The attack success rate (ASR) is used as the evaluation metric.
[0039] On LLaVA-1.5-7B, the attack success rates on the HADES dataset were: baseline model SD 77.3%, TYPO 73.7%, SD+TYPO 84.9%; after adopting the method in this application, these rates decreased to 12.2%, 9.40%, and 12.4%, respectively, with absolute reductions of 65.1, 64.3, and 72.5 percentage points. On the MM-SafetyBench dataset (containing 1680 text hints covering 13 harmful categories, each hint providing an explicitly harmful version): the baseline ASR was SD 80.6%, TYPO 79.6%, SD+TYPO 81.4%, which decreased to SD 12.6%, TYPO 8.00%, and SD+TYPO 11.7% after defense. The RedTeam2K dataset (containing 2000 explicit malicious text hints, covering 16 security policy categories) shows a baseline ASR of 39.3% for SD, 37.6% for TYPO, and 49.8% for SD+TYPO, which decreases to 19.0% for SD, 18.4% for TYPO, and 21.9% for SD+TYPO after defense. Compared with four baseline methods—AdaShield, ECSO, ShiftDC, and CMRM—the proposed method achieves best or near-best results across all nine evaluation lattices. Figure 5 As shown.
[0040] V. General Capability Validation. The impact of the proposed method on the general capability of the visual language model was evaluated on three benign benchmarks: MM-Vet, ScienceQA, and MME. MM-Vet evaluates six core visual language capabilities: recognition, OCR, knowledge, language generation, spatial perception, and mathematics, with a total of 218 questions. The ScienceQA test set contains 4241 multiple-choice questions, including 2224 plain text questions and 2017 multimodal questions. MME evaluates 14 sub-tasks of perception and cognition. On LLaVA-1.5-7B: MM-Vet decreased from 32.1 to 31.6 (a decrease of 0.50), ScienceQA remained unchanged at 64.0, and MME remained unchanged at 1754.9. This is because the normalized jailbreak-related offset of benign samples is usually less than 0.1, far below the preset threshold of 0.2, and most benign inputs will not trigger correction operations; even if they do, the adjustment amount is extremely small and will not significantly change the original representation. In comparison, AdaShield caused a 24.5 percentage point decrease in ScienceQA, CMRM caused a 1075.1 percentage point decrease in MME, and the generalizability of the method in this application was almost zero.
[0041] VI. Threshold τ ablation analysis. For example... Figure 4As shown, the impact of the threshold τ on security and general applicability is analyzed. On LLaVA-1.5-7B, when τ=1.0, it is equivalent to no correction, with HADES SD+TYPO ASR at 84.9% and ScienceQA at 64.0%; when τ=0.5, ASR drops to 51.8%, while ScienceQA remains at 64.0%; when τ=0.3, ASR drops to 16.4%, while ScienceQA remains at 64.0%; when τ=0.2, ASR drops to 12.4%, while ScienceQA remains at 64.0%; when τ=0.1, ASR drops to 11.8%, while ScienceQA drops to 61.1%; when τ=0.0, ASR drops to 11.4%, while ScienceQA drops to 59.3%. Decreasing τ from 1.0 to 0.2 significantly improves security without affecting general applicability. Further decreases in τ result in minimal marginal improvement in security while general applicability begins to decline. Therefore, τ=0.2 is the optimal value. This pattern also holds true for ShareGPT4V-7B (where HADES SD+TYPO ASR drops to 2.10% with τ=0.2, while ScienceQA remains unchanged at 62.7%) and InternVL-Chat-19B (where HADES SD+TYPO ASR drops to 6.10% with τ=0.2, while ScienceQA remains unchanged at 81.4%). τ=0.2 is a consistent value for all three visual language models. Empirically, the normalized jailbreak-related bias of jailbreak samples is usually greater than 0.3, while that of benign samples is usually less than 0.1. τ=0.2 falls between these two values and can reliably distinguish jailbreak input from benign input.
[0042] Example 2 In another implementation, the defensive effect of the method of this application in implicitly harmful scenarios and adversarial attack scenarios is further disclosed, verifying the cross-scenario generalization ability of the jailbreak direction vector.
[0043] I. Implicit Harmful Scenarios. In implicit harmful scenarios, the text prompts themselves are harmless; the harmful intent is conveyed solely through the image. HADES (Implicit Harmful): The original HADES dataset does not provide harmless text prompts. Using the corresponding harmless rewritten version provided by MML, along with SD+TYPO images (SD+TYPO images provide explicit textual guidance visually, helping visual language models understand the harmful semantics of image content), 750 implicit harmful samples were constructed. MM-SafetyBench (Implicit Harmful): Using the original harmless rewritten version of the dataset (Rephrased Question field), along with SD+TYPO images, 1680 implicit harmful samples were constructed.
[0044] On LLaVA-1.5-7B, the baseline ASR for HADES (implicit harmful substances) was 67.0%, which decreased to 35.3% after protection (a reduction of 31.7 percentage points); the baseline ASR for MM-SafetyBench (implicit harmful substances) was 67.2%, which decreased to 19.1% after protection (a reduction of 48.1 percentage points). On ShareGPT4V-7B, the baseline ASR for HADES (implicit harmful substances) was 66.8%, which decreased to 9.20% after protection (a reduction of 57.6 percentage points); the baseline ASR for MM-SafetyBench (implicit harmful substances) was 62.3%, which decreased to 12.9% after protection (a reduction of 49.4 percentage points). On InternVL-Chat-19B, the baseline ASR for HADES (Hidden Harmful Agents) was 67.7%, which decreased to 3.60% after defense (a decrease of 64.1 percentage points); the baseline ASR for MM-SafetyBench (Hidden Harmful Agents) was 47.7%, which decreased to 12.6% after defense (a decrease of 35.1 percentage points).
[0045] II. Adversarial Attack Scenarios. Evaluate the defense effectiveness under the following adversarial attack settings: MML Series Attacks: Apply rotation (MML-R), mirroring (MML-M), and Base64 encoding (MML-B64) transformations to the HADES image, preserving the original text prompts and modifying only the image, each comprising 2250 samples. HADES-Gradient: Iteratively optimize the HADES image using the gradient of LLaVA-1.5-7B to trigger a positive response, totaling 750 samples. SI-Attack: Apply a black-box scrambling text-image instruction permutation attack to HADES, selecting the most effective scrambling input based on feedback from a toxic evaluation model, totaling 2250 samples.
[0046] Defense effectiveness on LLaVA-1.5-7B: MML-M baseline 67.8%, after defense 9.00% (a decrease of 58.8 percentage points); MML-R baseline 63.7%, after defense 9.20% (a decrease of 54.5 percentage points); MML-B64 baseline 68.4%, after defense 9.60% (a decrease of 58.8 percentage points); HADES-Gradient baseline 76.1%, after defense 11.3% (a decrease of 64.8 percentage points); SI-Attack baseline 85.2%, after defense 14.9% (a decrease of 70.3 percentage points). On ShareGPT4V-7B: MML-M baseline 54.2%, 4.40% after defense; MML-R baseline 63.3%, 4.10% after defense; MML-B64 baseline 46.0%, 4.10% after defense; HADES-Gradient baseline 58.2%, 2.60% after defense; SI-Attack baseline 79.4%, 8.30% after defense. On InternVL-Chat-19B: MML-M baseline 24.5%, 10.4% after defense; MML-R baseline 27.7%, 9.40% after defense; MML-B64 baseline 30.6%, 11.8% after defense; HADES-Gradient baseline 31.4%, 7.60% after defense; SI-Attack baseline 68.9%, 9.30% after defense.
[0047] III. Generalization Ability Analysis. Although the jailbreak direction vectors were estimated only from the HADES explicit malicious dataset, they also achieved significant defensive effects in implicit malicious and adversarial attack scenarios, indicating that the jailbreak direction is an inherent structural feature of the model and does not depend on specific attack methods or data distributions. Further verification of the cross-dataset consistency of the jailbreak direction: From the HADES (H), MM-SafetyBench (M), and RedTeam2K (R) datasets, using three image types—SD (S), TYPO (T), and SD+TYPO (ST)—a total of nine jailbreak direction vectors were calculated. The cosine similarity between all pairs of vectors at layer 19 of LLaVA-1.5-7B exceeded 0.7, indicating that the jailbreak direction has high consistency across different datasets and image types, and is an inherent structural feature of the model rather than a product of specific data distributions.
[0048] Example 3 In another embodiment, the cross-model transfer verification of the method of this application on two visual language models of different sizes and architectures, ShareGPT4V-7B and InternVL-Chat-19B, is further disclosed.
[0049] I. ShareGPT4V-7B Implementation. ShareGPT4V-7B has the same size as LLaVA-1.5-7B (both with 7B parameters), but uses richer image-text alignment data for pre-training. For this model, only the jailbreak direction vector for the corresponding architecture needs to be re-estimated; the defense algorithm itself does not need to be modified. In explicitly hazardous scenarios: HADES SD+TYPO ASR decreased from 71.7% to 2.10% (a decrease of 69.6 percentage points), HADES SD decreased from 58.1% to 2.80% (a decrease of 55.3 percentage points), and HADES TYPO decreased from 55.1% to 2.40% (a decrease of 52.7 percentage points); MM-SafetyBench SD+TYPO decreased from 73.1% to 5.10% (a decrease of 68.0 percentage points), MM-SafetyBench SD decreased from 64.3% to 5.40% (a decrease of 58.9 percentage points), and MM-SafetyBench TYPO decreased from 65.1% to 4.20% (a decrease of 60.9 percentage points); RedTeam2K SD+TYPO decreased from 39.2% to 9.10% (a decrease of 30.1 percentage points). In terms of general capabilities: MM-Vet remains unchanged at 35.0 (decreased by 0.00), ScienceQA remains unchanged at 62.7% (decreased by 0.00), and MME remains unchanged at 1895.8 (decreased by 0.000), with no loss in general capabilities.
[0050] II. Implementation of InternVL-Chat-19B. InternVL-Chat-19B has 19B parameters, which is different from the previous two architectures. For this model, only the jailbreak direction vector of the corresponding architecture needs to be re-estimated; the defense algorithm itself does not need to be modified. In explicitly hazardous scenarios: HADES SD+TYPO ASR decreased from 31.7% to 6.10% (a decrease of 25.6 percentage points), HADES SD decreased from 38.9% to 5.60% (a decrease of 33.3 percentage points), and HADES TYPO decreased from 24.9% to 13.6% (a decrease of 11.3 percentage points); MM-SafetyBench SD+TYPO decreased from 73.1% to 5.70% (a decrease of 67.4 percentage points), MM-SafetyBench SD decreased from 71.7% to 4.40% (a decrease of 67.3 percentage points), and MM-SafetyBench TYPO decreased from 67.9% to 10.7% (a decrease of 57.2 percentage points); RedTeam2K SD+TYPO decreased from 43.6% to 9.60% (a decrease of 34.0 percentage points). In terms of general capabilities: MM-Vet decreased from 39.5 to 38.3 (a decrease of 1.20%), ScienceQA remained unchanged at 81.4% (a decrease of 0.00%), and MME remained unchanged at 2022.7% (a decrease of 0.000%), with almost no loss in general capabilities.
[0051] The experimental results of the three visual language models (LLaVA-1.5-7B, ShareGPT4V-7B, and InternVL-Chat-19B) consistently demonstrate that the method proposed in this application can effectively defend against various jailbreak attacks on visual language models of different scales (7B to 19B parameters), different architectures, and different pre-trained data, while maintaining almost zero loss in generality and exhibiting strong cross-model transferability.
[0052] It is understood that in the above embodiments, plain text input is constructed by removing images and corresponding visual inputs, retaining only text prompts, and processing them according to the model's plain text input method. The preset layer set is not limited to covering all decoding layers; it can also select a subset from intermediate to deep layers based on model characteristics, further balancing computational efficiency and defense effectiveness. The data source for estimating the jailbreak direction vector is not limited to explicit malicious datasets; it can also combine implicit malicious data or adversarial samples for multi-source estimation to adapt to the security requirements of specific deployment scenarios. The above descriptions are merely preferred embodiments of this application. For those skilled in the art, several modifications and improvements can be made without departing from the inventive concept of this application, and these all fall within the protection scope of this application.
Claims
1. A security defense method for jailbreak-related representation shift in a visual language model, characterized in that, Includes the following steps: S1: Construct a jailbreak sample set and a rejection sample set, including: collecting several multimodal harmful inputs, inputting the multimodal harmful inputs into a frozen visual language model to generate responses, judging each response as either a jailbreak or a rejection based on preset evaluation rules, and sampling several jailbreak samples and several rejection samples respectively; S2: Extract hidden state representations and estimate jailbreak direction vectors, including: inputting the jailbreak samples and rejection samples into the frozen visual language model respectively, extracting the hidden state representations of the last word positions in each layer of a preset layer set; for each preset layer, averaging the hidden state representations of the jailbreak samples to obtain the jailbreak quality vector. S1: Calculate the centroid of the rejection sample by averaging the hidden state representations of the rejection samples; calculate the difference between the escape centroid and the rejection centroid, and perform L2 normalization on the difference vector to obtain the escape direction vector; the escape direction vector remains fixed in subsequent inference; S2: Calculate the hidden state representation of the last word of the multimodal input, including: for the multimodal input to be inferred, the multimodal input includes images and text prompts, and without enabling intervention, input the multimodal input into the visual language model to extract the hidden state representation of the last word position in each layer of the preset layer set; S3: Construct a plain text input and calculate the total table of image introduction. The image-introduced representation offset includes: constructing a plain text input corresponding to the multimodal input; inputting the plain text input into the visual language model without intervention; extracting the hidden state representations of the last word positions at the same layer; calculating the difference between the hidden state representations of the multimodal input and the plain text input at each layer to obtain the total representation offset introduced by the image; S5: calculating a jailbreak-related offset scalar value, including: projecting the total representation offset introduced by the image onto the jailbreak direction vector to obtain the jailbreak-related offset scalar value; S6: calculating the normalized jailbreak-related offset and determining whether to perform correction, including: normalizing the jailbreak-related offset... The offset scalar value is divided by the L2 norm of the total representation offset to obtain the normalized jailbreak-related offset; at the first decoding step, it is determined whether the normalized jailbreak-related offset exceeds a preset threshold; S7: Representation correction and security reasoning, including: performing intervention-based generation forward propagation on the original multimodal input; if the normalized jailbreak-related offset exceeds the preset threshold, the product of the jailbreak-related offset scalar value and the jailbreak direction vector is subtracted from the hidden state representation of the last word in the corresponding layer of this forward propagation to obtain the corrected hidden state representation; based on the corrected hidden state representation, autoregressive generation continues, and a security response is output.
2. The jailbreak-related representation shift security defense method for a visual language model according to claim 1, characterized in that, In step S1, the preset evaluation rule adopts a consistency screening strategy of three methods: keyword matching, evaluation by the first language model, and evaluation by the second language model. Only samples with completely consistent evaluation results from the three methods are retained, while samples with conflicting evaluation results are discarded.
3. The jailbreak-related representation shift security defense method for a visual language model according to claim 1, characterized in that, In step S1, the sample selection for estimating the jailbreak direction vector adopts a conflict-discarding strategy, retaining only samples whose judgment results are completely consistent across the three methods of keyword matching, first language model evaluation, and second language model evaluation, in order to reduce annotation noise; when evaluating the defense performance, a majority voting strategy is used to determine the response, and the response is labeled as jailbreak only when at least two methods consistently determine it as jailbreak.
4. The jailbreak-related representation shift security defense method for a visual language model according to claim 1, characterized in that, In step S1, both the jailbreak samples and the refusal-to-answer samples come from the same explicit harmful dataset, in which the text prompts themselves clearly contain harmful intent. The jailbreak direction vector, estimated from explicit malicious datasets, can be directly generalized to implicit malicious scenarios and adversarial attack scenarios.
5. A security defense method for jailbreak-related representation shift in a visual language model according to claim 1, characterized in that, The formula for calculating the jailbreak direction vector in step S2 is as follows: ,in Let the vector be the difference between the centroids of the escape and the centroids of the refusal to answer. and They represent the first The centroids of the hidden states of the escaped and rejected samples at each level. For layer index, For the first The escape direction vector of the layer, The L2 norm is represented; the escape direction vector is a unit vector pointing from the rejection centroid to the escape centroid, representing the internal transition direction from the rejection state to the escape state in the model representation space.
6. A security defense method for jailbreak-related representation shift in a visual language model according to claim 1, characterized in that, In step S4, the plain text input is constructed by removing the image and corresponding visual input, retaining only the text prompt, and processing it according to the model's plain text input method; the formula for calculating the total representation offset is: ,in For the first The total representation offset vector introduced by the layer image. For layer index, For multimodal input The hidden state representation of the last word element, Represents an image. This indicates a text prompt. For corresponding plain text input The hidden state representation of the last word element, This indicates that it does not contain images and corresponding visual input.
7. A security defense method for jailbreak-related representation shift in a visual language model according to claim 1, characterized in that, The formula for calculating the jailbreak-related offset scalar value in step S5 is as follows: ,in For the first The jailbreak-related offset scalar value of the layer. Indicates transpose. The total representation offset vector introduced for the image. For the first The escape direction vector of the layer. For layer index; the calculation formula for the normalized jailbreak-related offset in step S6 is as follows: ,in For the first The normalized jailbreak-related offset of the layer, The L2 norm is represented, and the normalized jailbreak-related offset ranges from [-1, 1].
8. A security defense method for jailbreak-related representation shift in a visual language model according to claim 6, characterized in that, The calculation formula for the correction in step S7 is as follows: , ,in For the revised first Hidden state representation of end-layer lexical units, For multimodal input The hidden state representation of the last word element, For the first The offset scalar value related to the layer jailbreak. For the first The escape direction vector of the layer. For layer index; For the first The normalized jailbreak-related offset of the layer, when The correction is performed in time, where For a preset threshold, the correction is to subtract the projection component of the total representation offset introduced by the image in the escape direction from the multimodal hidden state.
9. A security defense method for jailbreak-related representation shift in a visual language model according to claim 1, characterized in that, The preset layer set covers all decoding layers of the visual language model; steps S5 and S6 are calculated on each preset layer respectively, and step S7 is executed according to the layer order of the model forward propagation. On each layer, the jailbreak-related offset scalar value and normalized jailbreak-related offset of that layer are calculated respectively, and the hidden state representation of that layer is corrected when the threshold condition is met.
10. A security defense method for jailbreak-related representation shift in a visual language model according to claim 1, characterized in that, The calculation of the total representation offset introduced by the image in steps S3 and S4 is completed before the first decoding step of the inference, requiring two uninterrupted reference forward propagations; then, an intervened generative forward propagation is performed on the original multimodal input, and autoregressive generation continues; in step S7, the representation correction is only performed once in the pre-filling stage before the generation of the first output lexical, and the correction is not applied again in subsequent decoding.
