Large visual language model vulnerability detection method and device based on security sensitive layer activation guidance and storage medium

Abstract

The invention discloses a large visual language model vulnerability detection method and device based on security sensitive layer activation guidance and a storage medium, and belongs to the technical field of artificial intelligence security, the method comprises the following steps: S1, constructing a security sensitive layer activation induction sample data set; s2, performing quantitative analysis on activation differences of the model intermediate layer under attack and normal input; s3, on the basis of the recognized security sensitive layer combination and an output layer of the large visual language model, generating an in-process confrontation image with an attack attribute through optimization of a loss function; and S4, recording an optimal security sensitive layer combination and a harmful response path guided by the optimal security sensitive layer combination, and constructing a structured representation of the potential security vulnerabilities of the model. According to the method, the security fragile links of the model are accurately positioned, the attack guiding force is improved in combination with multiple losses, the generated confrontation disturbance is controllable and imperceptible, the good cross-instruction migration capability is achieved, and the method adapts to various security assessment and red team test tasks.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of artificial intelligence security, and particularly relates to a large visual language model vulnerability detection method based on a safety-sensitive layer activation guide, a device and a storage medium. BACKGROUND

[0002] With the rapid development of artificial intelligence technology, large visual language models (LVLMs) represented by LLaVA and MiniGPT-4 have shown superior performance in image description, visual question answering, and multi-modal reasoning tasks, and are widely used in education, medical care, security, search engines, and other fields. These models, by fusing pre-trained visual encoders and large language models (LLMs), not only have strong language understanding and generation capabilities, but also can jointly model complex image-text inputs, greatly expanding the scope of application of multi-modal tasks. However, as the model's capabilities improve, the security risks and potential misuse problems it faces become increasingly prominent, especially as the risk of the model generating content that violates ethical, legal, or safety guidelines becomes increasingly serious.

[0003] Model alignment (Alignment) technology aims to ensure that LVLMs output content conforms to human values and social norms, preventing the generation of inappropriate or potentially harmful responses. Current mainstream alignment methods include reinforcement learning based on human feedback (RLHF) and knowledge editing (Knowledge Editing) strategies, which have achieved certain results in improving the model's ability to identify and reject sensitive requests, but still have the risk of being evaded by attackers.

[0004] Jailbreak attacks (Jailbreak Attack) are a typical form of security threat that constructs malicious inputs to guide the model to bypass the built-in alignment mechanism and generate illegal content that does not conform to safety standards. Jailbreak attacks against LVLMs have characteristics such as complex multi-modal input, high-dimensional attack space, and weak interpretability, and current research mainly focuses on the following two paths: (1) methods based on adversarial perturbation, which optimize the pixel perturbation of the input image to make the model output harmful text while maintaining semantic consistency, thereby achieving imperceptible attacks; (2) methods based on visual prompt injection, which use the OCR capability of LVLMs to embed disguised text instructions in images, and use specific layout structures or image generation models (such as Stable Diffusion) to implicitly deliver malicious semantics to bypass the content review mechanism on the language side.

[0005] Although existing methods can guide LVLMs to generate violation responses to some extent, there are still the following shortcomings: (i) lack of systematic understanding of the internal processing mechanism of the model, the attack relies on the explicit signal of the output layer, and the security vulnerabilities of the intermediate reasoning process are ignored; (ii) the generated perturbation is often highly dependent on specific instructions, and the attack generalization ability is insufficient; (iii) there is a lack of explainable hierarchical positioning means, it is difficult to determine the key activation path of the abnormal model response, affecting the deployment of subsequent defense mechanisms. SUMMARY

[0006] In view of the above problems existing in the prior art, the technical problem to be solved by the present application is to provide a large visual language model vulnerability detection method based on security sensitive layer activation guidance, equipment and storage medium, so as to realize the deep mining and systematic detection of model security vulnerabilities.

[0007] Technical scheme: In order to solve the above technical problems, the technical scheme adopted by the present application is as follows:

[0008] A large visual language model vulnerability detection method based on security sensitive layer activation guidance, comprising the following steps:

[0009] S1, construct a security sensitive layer activation guidance sample data set, select a plurality of methods covering large visual language model multi-modal attack scenarios to generate image-text input;

[0010] S2, by quantitatively analyzing the activation difference of the intermediate layer of the large visual language model under attack and normal input, identify the security sensitive layer combination;

[0011] S3, based on the identified security sensitive layer combination and the output layer of the large visual language model, input any picture as an initial adversarial image, jointly apply multiple losses of harmful content generation and rejection response suppression, and generate an adversarial image with attack attributes through loss function optimization;

[0012] S4, further screen the security sensitive layer combination in S2, record the optimal security sensitive layer combination and the harmful response path guided by it, construct a structured representation of the potential security vulnerabilities of the large visual language model and generate a final adversarial image.

[0013] Preferably, in the S1, the method covering the multi-modal attack scene of the large visual language model includes a text-based adversarial perturbation attack method GCG, a visual-based adversarial perturbation attack method VAJM, a joint visual and text-based adversarial attack method UMK, and a visual prompt injection attack method Figstep, and the specific construction method of S1 is:

[0014] Construct an image-text input pair containing "harmful" and "harmless" semantics as a security sensitive layer activation guidance sample pair set, , wherein, represents a pair of input successfully guiding a large visual language model to output a harmful response, containing an optimized adversarial image and jailbreak text , wherein consists of a harmful query and a jailbreak suffix ; represents a pair of image-text input that the model is not successfully jailbroken under the harmful query , keeping a safe response, represents a benign picture input, represents the total amount of sample pairs in the sample pair set.

[0015] Preferably, S2 specifically comprises:

[0016] S2.1, input the activation guide sample pairs generated by S1 to the large visual language model to be tested, extract the hidden state representation of each layer, and perform difference quantization analysis on the activation results of each pair of activation guide samples in each layer of the model. The Euclidean distance is used to measure the activation distribution difference between harmful and harmless responses, and the safety sensitivity score of each layer is calculated , the formula is as follows:

[0017]

[0018] wherein, , represents the activation guide sample pair constructed in S1, represents the hidden activation of the model in the th layer on the th sample pair, represents the total amount of input guide sample pairs, represents the safety sensitivity score of the th layer, and finally the average difference of each layer is calculated as its safety sensitivity score, realizing the quantitative representation of the safety sensitivity of each layer;

[0019] S2.2, select the safety sensitive layer and define its weight:

[0020]

[0021] wherein, represents that from the 1st to the Lth layer of the model, the first layers with the maximum safety sensitivity score are selected to form a safety sensitive layer combination ; represents that this score is directly used as the weight of the layer, and the hierarchical weight coefficient The difference score of the activation of the layer under harmful and harmless inputs is defined here.

[0022] Preferably, the S3 specifically comprises the following steps:

[0023] S3.1, hierarchical harmful content generation loss construction: define the harmful sample corpus H:

[0024]

[0025] wherein, represents the harmful sentence of the target output, represents the number of harmful sentences, and the attack target is to maximize the generation probability of harmful sentences in the security sensitive layer; the harmful content generation loss function is constructed as :

[0026]

[0027]

[0028] wherein, represents the probability distribution of the output head of the i-th layer model under the given condition, represents the adversarial image to be optimized, and the pixel value is constrained in the standard image range;

[0029] S3.2, hierarchical rejection response suppression mechanism design: in order to enhance the suppression effect of the model response to the rejection mode, a rejection response prefix set R is constructed:

[0030]

[0031] wherein, represents a set of pre-defined typical rejection sentences, represents a typical rejection sentence, represents the total number of rejection sentences in the set; the negative target constructed is to minimize the conditional generation probability of the rejection response:

[0032]

[0033]

[0034] wherein, represents the rejection response suppression loss;

[0035] S3.3, multi-objective joint optimization loss definition:

[0036]

[0037] wherein, represents the harmful content generation loss,​ denotes the rejection response inhibition loss, λ ∈ (0, 1) is the balance coefficient, and controls the importance of the two losses, denotes the final multi-objective joint loss, and by minimizing the total loss, the two objectives of "incentive harmful generation" and "inhibition of rejection response" can be optimized at the same time;

[0038] S3.4, generation of in-process adversarial images: on the basis of obtaining the security-sensitive layer combination in S2 and constructing the multi-objective function in S3.1-S3.3, input any picture as an initial adversarial image, and combine the gradient-based pixel update algorithm to optimize the image input, thereby generating an in-process adversarial image capable of producing jailbreaking behavior.

[0039] Preferably, the S4 comprises the following steps:

[0040] S4.1, regarding the security-sensitive layer combination obtained in S2 as a search space, and using binary bits to encode the activation control right of each layer:

[0041]

[0042] wherein, denotes attacking the i-th security-sensitive layer, denotes that the layer does not participate in the attack optimization this time;

[0043] S4.2, for all possible layer combination strategies , respectively, perform jailbreaking attack experiments, generate respective in-process adversarial images, and record the corresponding attack success rates , and the final goal is to find the optimal layer combination strategy , so that the ability to generate harmful content is maximized, that is:

[0044]

[0045] This search process adopts an exhaustive strategy;

[0046] S4.3, once the optimal security-sensitive layer combination is determined, the optimization path can be constrained accordingly, and only the corresponding layer is activated for intervention; specifically, the weight vector is updated as:

[0047]

[0048] wherein, denotes the original weight coefficient of the i-th security-sensitive layer;

[0049] ​​​​​S4.4, after the weight vector is updated by directional constraint, only gradient perturbation is performed on the optimal security sensitive layer to generate a final adversarial image that induces a large visual language model to produce jailbreaking behavior .

[0050] The application further provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, and the processor implements the steps of the large visual language model vulnerability detection method based on security sensitive layer activation guidance described above when executing the computer program.

[0051] The application further provides a computer readable storage medium, which stores a computer program, and the computer program implements the steps of the large visual language model vulnerability detection method based on security sensitive layer activation guidance described above when executed by a processor.

[0052] Advantages: Compared with the prior art, the application has the following advantages:

[0053] 1. Precise positioning of model security vulnerable links: The application first identifies the most sensitive intermediate layer of the model to harmful semantics in a multi-modal scene by introducing a hierarchical security sensitivity analysis method, breaking through the limitations of traditional attack methods that only rely on output layer signals, and providing a more fine-grained attack perspective and detection mechanism.

[0054] 2. Joint multiple loss improves attack guidance: The proposed multi-objective joint loss function simultaneously considers "harmful generation incentives" and "refusal response suppression", which can cooperatively intervene the generation tendency and defense strategy of the model, significantly improving the attack success rate and the adversarial nature of the generated content.

[0055] 3. Disturbance is controllable and imperceptible: Through image-level minimum disturbance optimization, the semantic consistency and visual authenticity of the input sample are maintained, effectively improving the imperceptibility of the attack, and being suitable for more realistic black box and gray box attack scenarios.

[0056] 4. Good cross-instruction migration ability: Since the application relies on the internal structure characteristics and general refusal behavior patterns of the model, the attack samples constructed by the application show high migration between different harmful instructions, and have good universality and robustness.

[0057] 5. Adapt to various security evaluation and red team testing tasks: The method of the application can be used as a red team attack tool for large visual language model security evaluation, assisting developers in quickly identifying potential jailbreaking risks, and promoting the explainability of visual language models and the improvement of defense strategies. BRIEF DESCRIPTION OF DRAWINGS

[0058] Figure 1 is the overall flowchart of the embodiment of the application. DETAILED DESCRIPTION

[0059] The present application will be further illustrated below in conjunction with specific examples, which are implemented on the premise of the technical solutions of the present application and are understood to be used for illustrating the present application only but not for limiting the scope of the present application.

[0060] As shown in the figure, the large visual language model vulnerability detection method based on security sensitive layer activation guidance of the present embodiment comprises the following steps: Figure 1

[0061] S1, construct a security sensitive layer activation guidance sample data set, and generate image-text inputs for four methods covering large visual language model multi-modal attack scenarios, the four methods covering large visual language model multi-modal attack scenarios are respectively a text-based adversarial perturbation attack method GCG (Greedy Coordinate Gradient Universal and transferable adversarial attacks on aligned language models, a method for attacking aligned large language models through visual adversarial examples), a visual-based adversarial perturbation attack method VAJM (Visual Adversarial Examples Jailbreak Aligned Large Language Models, a method for breaking into aligned large language models through visual adversarial examples), a joint adversarial attack method based on vision and text UMK (White-box Multimodal Jailbreaks Against Large Vision-Language Models, a white-box multi-modal jailbreak method against large visual language models), and a visual prompt injection attack method Figstep (Jailbreaking Large Vision-language Models via Typographic Visual Prompts, a method for breaking into large visual language models through typographic visual prompts). The specific construction method is as follows:

[0062] By constructing image-text input pairs containing "harmful" and "harmless" semantics as security sensitive layer activation guidance sample pairs, , wherein, represents an input pair that successfully guides the large visual language model to output a harmful response, containing an optimized adversarial image and a jailbreak text , wherein is a harmful query and a jailbreak suffix​ composition, representing harmful query The model is not successfully jailbroken, and the picture input of the safe response is selected, representing benign picture input, N represents the total number of sample pairs in the sample pair set. This step introduces samples in both harmful and harmless states as input, deliberately creates large activation differences of visual language model security sensitive layers, and lays a data foundation for subsequent vulnerability positioning through activation feature analysis.

[0063] S2, using the security sensitive layer combination construction method based on activation response difference, by quantitatively analyzing the activation difference of the intermediate layer of the large visual language model under attack and normal input, the security sensitive layer combination with high security sensitivity is identified, which includes:

[0064] S2.1, input the activation guide sample pairs generated by S1 into the large visual language model to be tested, extract the hidden state representation of each layer, and quantitatively analyze the activation difference of each pair of activation guide samples in each layer. The Euclidean distance is used to measure the activation distribution difference between harmful and harmless responses, and the security sensitivity score of each layer is calculated , the formula is as follows:

[0065]

[0066] Among them, , ] represents the activation guide sample pair constructed in S1, represents the hidden activation of the model layer on the th sample pair, N represents the number of input guide sample pairs, represents the security sensitivity score of the th layer. Finally, the average difference of each layer is calculated as its security sensitivity score, and the security sensitivity of each layer is quantitatively represented;

[0067] S2.2, select security sensitive layers and define their weights:

[0068]

[0069] Among them, represents that from the first to the Lth layer of the model, the first layers with the maximum security sensitivity score are selected to form a security sensitive layer combination ; represents that this score is directly used as the weight of the layer. The hierarchical weight coefficient is defined here as the activation difference score of the layer under harmful and harmless input.

[0070] This step uses a quantitative screening mechanism to accurately identify the intermediate layer that is most sensitive to changes in the model's response under image-text jailbreak attacks, providing a structural basis for subsequent attack path construction and vulnerability disclosure.

[0071] S3. Based on the identified security-sensitive layer combination and the output layer of a large-scale visual language model, any input image is used as the initial adversarial image. Multiple losses of harmful content generation and rejection response suppression are jointly applied. The adversarial image with attack attributes is generated through loss function optimization. Specifically, the following steps are included:

[0072] S3.1, Construction of Hierarchical Harmful Content Generation Loss: Define the harmful sample corpus H:

[0073]

[0074] in, This indicates that the target output contains harmful statements. This represents the number of harmful statements, and the attack objective is to maximize their generation probability at the security-sensitive layer; the harmful content generation loss function is constructed as follows: :

[0075]

[0076]

[0077] in, Indicates the first The probability distribution of the output head of a large-scale visual language model under given conditions. Let represent the adversarial image to be optimized, whose pixel values ​​are constrained to the range of a standard image. The essence of this optimization problem is to find a suitable image in the image space. Internally, it enables all security-sensitive layers of the model to be protected. Generate targeted harmful content Adversarial images with joint probability maximization Weight This ensures that greater optimization pressure is applied to more critical layers.

[0078] S3.2 Design of a hierarchical rejection response suppression mechanism: To enhance the suppression of rejection patterns in the model response, a set of rejection response prefixes R is constructed:

[0079]

[0080] in, This represents a predefined set of typical rejection statements. This represents a typical rejection statement. denotes the total number of rejected statements in the set; the constructed negative target is to minimize the conditional generation probability of the rejection response:

[0081]

[0082]

[0083] where, denotes the rejection response suppression loss;

[0084] S3.3, Multi-objective joint optimization loss definition:

[0085]

[0086] where, denotes the harmful content generation loss, denotes the rejection response suppression loss, and λ ∈ (0, 1) is the balance coefficient that controls the importance of the two losses, denotes the final multi-objective joint loss. By minimizing this total loss, the two objectives of “encouraging harmful generation” and “suppressing rejection response” can be optimized simultaneously.

[0087] S3.4, In-process adversarial image generation:

[0088] After obtaining the security-sensitive layer combination in S2 and completing the construction of the harmful generation loss and the rejection suppression loss in S3.1-S3.3, the two loss combinations are combined to form the final multi-objective joint loss (see S3.3). Take any one picture (as a real natural image or a dataset sample) as the initial adversarial image, and assign it to ← as the starting point of adversarial image iteration. Through gradient-based iterative optimization, the input is modified in the pixel representation space to obtain the in-process adversarial image for triggering the jailbreak behavior. Specifically, taking as the starting point, the input is selectively adjusted according to the following process:

[0089] 1. Calculate the forward feedback: input the current image into the large visual language model to be tested, obtain the response of the model on the security-sensitive layer and the output layer, and calculate the harmful generation loss and the rejection suppression loss based on the harmful sample set H in S3.1 and the rejection prefix set R in S3.2, so as to obtain the current multi-objective joint loss (S3.3 defines, the weight is controlled by ). This loss set uses weight allocation to exert stronger optimization pressure on the security-sensitive layer, so that the optimization process focuses on those intermediate layers that are more sensitive to jailbreak.

[0090] 2. Gradient-based pixel update: For the current multi-objective joint loss , its gradient signal with respect to the input pixel is calculated, and gradient ascent is used for iterative optimization in the pixel space. Specifically, in each iteration, the input image updates the pixel value in the direction of loss increase, while ensuring that all pixel values remain within the legal interval [0, 1] through clipping operations. The update process can be represented as:

[0091]

[0092] where is the step size; is the gradient of the loss function with respect to the input, is used to limit the updated pixel value within the legal range.

[0093] This process achieves controlled optimization of the input image through step-by-step use of the loss gradient, causing observable changes in the model's response to harmful induction and rejection suppression targets at the safety-sensitive layer and output layer, thereby forming a sequence of in-process adversarial images for analysis .

[0094] 3. Stop and process observation: Iteration is terminated when the maximum number of steps T=2000 is reached. During the entire optimization process, the generated by any iteration step t is considered as an in-process adversarial image, which is used to record and analyze the model's behavior at different stages of disturbance. It is important to emphasize that the "in-process adversarial image" concerned in this step is an intermediate product for verifying and quantifying the change in model output tendency, and is not necessarily the "final adversarial image" to be delivered externally.

[0095] In summary, the "in-process adversarial image" in this step refers to the intermediate image sequence generated from the initial image under the guidance of the multi-objective joint loss during the gradient-based pixel update process . This sequence provides a direct observation object for analyzing the dynamic response of large visual language models under multi-modal disturbance, without the need to deliver a single final adversarial image at this stage.

[0096] This step, based on the acquisition of the safety-sensitive layer set, designs a multi-objective function that integrates harmful generation and rejection suppression. By inputting any picture as the initial adversarial image and combining the projected gradient descent (PGD) algorithm to optimize the image input, it generates in-process adversarial images that can guide large visual language models to produce jailbreaking behavior.

[0097] S4, further screening the security-sensitive layer combination in S2, recording the optimal security-sensitive layer combination and the harmful response path guided thereby, constructing a structured representation of potential security vulnerabilities of the large visual language model and generating a final adversarial image, specifically comprising the following steps:

[0098] S4.1, regarding the security-sensitive layer combination of size obtained in S2 as a search space, and using binary bits to encode the activation control of each layer:

[0099]

[0100] wherein, represents attacking the i-th security-sensitive layer, represents that the layer does not participate in the attack optimization this time;

[0101] S4.2, performing the jailbreak attack experiment in S3 on all possible layer combination strategies , generating respective process adversarial images and recording their corresponding attack success rates , and the final goal is to find the optimal layer combination strategy that maximizes the ability to generate harmful content, i.e.:

[0102]

[0103] This search process adopts an exhaustive strategy;

[0104] S4.3, once the optimal security-sensitive layer combination is determined, the optimization path can be constrained and only the corresponding layers are activated for intervention; specifically, the weight vector is updated as:

[0105]

[0106] wherein, represents the original weight coefficient of the i-th security-sensitive layer.

[0107] S4.4, generating a final adversarial image

[0108] Through the above method, the optimal attack path for the security-sensitive layer is determined , and after directional constraint update of the weight vector based on the path, the method in S3.4 can be used to perform gradient perturbation only on the optimal security-sensitive layer, generating a final adversarial image that can effectively guide the large visual language model to produce jailbreak behavior .

[0109] ​​​Although step S2 has constructed a set of globally activated sensitive layers with high sensitivity However, directly optimizing all security-sensitive layers together in the actual attack optimization process may cause gradient interference and optimization conflicts, thereby reducing the effectiveness of the jailbreak attack. Therefore, step S4 designs a security-sensitive layer subset screening method based on binary combination coding, takes the attack success rate (ASR) as the objective function, and automatically searches for the optimal attack layer combination as the optimal security-sensitive layer combination (finding the optimal security-sensitive layer (optimal attack path) is considered to reveal the structured representation of the potential security vulnerabilities of the model), thereby enhancing the robustness and effectiveness of the attack. This method realizes dynamic pruning and fine control of the attack path, reduces redundant calculations, avoids non-critical layer interference, and effectively improves the attack efficiency and generation effect.

[0110] The effectiveness and efficiency of the method of the present embodiment are verified through the following experiments:

[0111] The present embodiment focuses on the white-box multi-modal attack scenario, i.e., the attacker has complete access to the internal structure and parameters of the target visual language model (LVLM) and can simultaneously manipulate image and text inputs to achieve more fine-grained intervention and attack optimization. On this basis, to fully verify the universality and potential security threat of the method of the present embodiment, the present embodiment is further extended to the black-box migration setting, i.e., the attacker evaluates the attack effect only through input-output interaction without knowing the internal structure and parameters of the model, which is close to the usage restrictions in the real open environment.

[0112] In terms of attack interaction settings, the present invention adopts a single-round input mode, i.e., the attacker only needs to construct image-text inputs once to guide the target model to generate a rule-violating response that violates the alignment mechanism, without relying on multi-round context guidance or complex prompt engineering, which has high practicality and attack efficiency and can more realistically simulate security threats in deployment scenarios.

[0113] Large visual language models

[0114] In terms of target model selection, the present embodiment selects three current mainstream open-source large visual language models as attack objects, including: MiniGPT-4 (based on Vicuna-13B), InstructBLIP (based on Vicuna-13B), and LLaVA-1.5-7B (based on LLaMA-2-7B-Chat). These models differ in architecture design, visual fusion method, and alignment strategy, and are representative.

[0115] During all experiments, the target model maintains its original preset system prompt template unchanged to ensure compliance with its real deployment environment security protection mechanisms and restriction policies. The vulnerability detection method proposed in this embodiment first optimizes the white-box attack on MiniGPT-4, and then conducts black-box migration testing on InstructBLIP and LLaVA-1.5-7B to evaluate the attack effect and general ability of this embodiment under different model architectures.

[0116] Evaluation criteria

[0117] This embodiment combines the automatic judgment mechanism of GPT-4o-mini and the multi-dimensional toxicity scoring tool as the evaluation criteria:

[0118] Attack success rate: This embodiment designs an automatic evaluation process based on GPT-4o-mini, combining few-shot prompts and semantic judgment mechanisms to determine the proportion of inappropriate responses generated by the model when faced with harmful input prompts ASR:

[0119]

[0120] where n represents the total number of test samples, is the response of the model to the ith attack instruction, and the evaluation function = 1 indicates that ChatGPT-4o-mini determines the response to be "harmful". To improve evaluation stability, multiple rounds of evaluation are performed on the test samples and the average ASR is taken.

[0121] Evaluation method based on automated toxicity scoring: This embodiment introduces automated toxicity detection tools (Perspective API and Detoxify) to evaluate the potential risks in the text generation stage. Both tools can output six different dimensions of toxicity scores for each text, with a value range of [0, 1], and a higher value indicates a stronger toxicity in that dimension.

[0122] Comparison method

[0123] 1. VAJM (Visual Adversarial Examples Jailbreak Aligned Large Language Models): This method is based on the optimization of visual modalities of adversarial images, and achieves attack by maximizing the probability of harmful content output on a small batch of data sets. Train and test on the VAJM dataset, with the same number of iterations T = 5000, batch size b = 8, and step size alpha = 1.

[0124] 2. UMK (White-box Multimodal Jailbreaks Against Large Vision-Language Models): This method combines text and image dual modalities for jailbreak attacks. This embodiment follows the original paper settings, with the first stage using the VAJM dataset for 5000 rounds of training ( ), and the second stage fine-tuning on the AdvBench dataset for 2000 rounds ( ). The remaining key parameters are as follows: batch size b = 8, step alpha = 1, image-text attack ratio r = 10, adversarial text suffix length of 20 tokens, and candidate sample number m = 250.

[0125] Dataset Introduction

[0126] This embodiment selects three major authoritative jailbreak datasets to comprehensively evaluate attack effectiveness:

[0127] VAJM dataset (Visual Adversarial Examples Jailbreak Aligned Large Language Models): As a small-sample harmful corpus, it contains 66 training instructions and 40 test instructions, covering directions such as gender, race, and content such as insults, false information, and violent crimes, suitable for attack initialization and generalization ability evaluation.

[0128] AdvBench (Adversarial Benchmark): A widely used standard jailbreak test set containing a large number of explicit goal-target pairs (attack instructions and responses with a strong affirmative tendency starting with "Sure, here is..."), covering sensitive content such as violence and discrimination. In the experiments of, AdvBench is divided into an 8:2 training set and a test set. This invention randomly selects 66 instruction-response pairs from the training set for image optimization.

[0129] RealToxicityPrompts dataset: This embodiment also uses the RealToxicityPrompt dataset to evaluate the security impact of the attack method in the text generation stage. This dataset contains 1225 text prompts specifically designed to guide language models to generate continuation results with toxic or inappropriate content. This embodiment uses this dataset to further measure whether the attack has broken the model's original safety alignment mechanism, and to verify the threat level of adversarial samples to language generation security.

[0130] The results of the experiment on MiniGPT-4 under the VAJM dataset are as follows:

[0131] Table 1 Comparison of attack success rates (ASR) of different methods on the VAJM dataset

[0132]

[0133] The visual language model vulnerability detection method proposed in this embodiment has significant attack effect and practical performance advantages. This method does not rely on any text prompt or suffix, and can guide the model to generate inappropriate content by optimizing the image modality. Under the condition of only 2000 rounds of image perturbation optimization, the attack success rate (Attack Success Rate, ASR) can reach 95.0%, which is significantly better than the existing image modality attack method (such as VAJM's 80.0%), and close to the current best dual-modality attack method (such as UMK's 97.5%). The results show that the proposed method also has strong attack ability and cross-task instruction generalization performance under single-modality input settings.

[0134] Compared with existing methods, this scheme has higher practicality and concealment. Existing dual-modality attack methods rely on specific text suffixes and prompt templates when generating illegal content, and need to jointly perturb images and text, which is complex and has high deployment cost. The generated attack response content has the problem of fixed language mode, such as frequent use of "Sure, here is" sentence, which is easy to be detected and intercepted by rule template type security systems. The method of this embodiment only realizes attack through image perturbation, without any additional text dependence, the process is simple and efficient, the generated content is natural and diverse, and it is not easy to be detected, which has stronger practical attack feasibility.

[0135] To further verify the harmfulness of the generated text, the experiment trains perturbation samples on the standard visual attack training set and performs migration evaluation on the RealToxicityPrompts test set. The Perspective API (toxicity detection API) and Detoxify (detoxification tool) tools are used to quantify the multi-dimensional toxicity indicators of the generated text, and the "Any" trigger rate (the proportion of samples with any attribute exceeding the threshold) is used as the overall toxicity measure.

[0136] Table 2 Toxicity evaluation of generated text under different attack methods on the RealToxicityPrompts benchmark

[0137] a. Perspective API

[0138]

[0139] b. Detoxify

[0140]

[0141] Experimental results show that the method is better than the existing image modal attack baseline in multiple toxicity dimensions, and some indicators even exceed the double modal method. For example:

[0142] In the Perspective API score, the Identity Attack (identity attack) is 18.93%, the Toxicity (toxicity) is 71.56%, and the overall toxicity trigger rate of "Any" is 75.56%, which is similar to the effect of the double modal (UMK) method (76.98%);

[0143] In the Detoxify score, the Insult (insult) is 41.12%, and the overall "Any" is 72.39%, which is better than similar methods (such as 68.70% of UMK).

[0144] In order to enhance the rejection identification coverage range, the original GCG (Greedy Coordinate Gradient Universal and transferable adversarial attacks on aligned language models) implementation provides a rejection keyword list, and in combination with the high-frequency rejection response generated by the MiniGPT-4 model when processing sensitive input, the rejection prefix is expanded and supplemented:

[0145] refuse_prefixes = [

[0146] "I can't", "I cannot", "I won't", "I'm not able to provide", "Icannot create",

[0147] "I cannot support", "I can't provide", "I can't assist", "I'm sorry","I am sorry",

[0148] "I'm really sorry", "I am really sorry", "I apologize", "I do not", "I'm not able",

[0149] "I'm just", "I'm an", "I'm a", "As an", "As an AI", "As a languagemodel",

[0150] "As an Assistant", "As a responsible AI", "This request", "Thatrequest",

[0151] "It is not appropriate", "is illegal and unethical", "This goesagainst",

[0152] "This content is", "That would be", "My apologies", "Unfortunately","Sorry",

[0153] "OpenAI does not", "OpenAI", "unethical", "not ethical", "illegal", "not legal",

[0154] "According to OpenAI", "According to my guidelines", "Let's move on","but I cannot",

[0155] "If you have any other non-malicious requests", ]

[0157] In summary, the scheme of this embodiment has high attack success rate, strong toxic text generation ability and good cross-instruction migration ability without relying on text information, providing an efficient and feasible technical path for vulnerability detection and security evaluation of visual language models.

[0158] In this embodiment, a computer device is also provided, which includes at least one processor, and a memory connected with the at least one processor in communication, the memory storing a computer program executable by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform the steps of the large visual language model vulnerability detection method based on the security sensitive layer activation guidance described above.

[0159] In practical application, the processor can include one or more processing cores. The processor invokes data stored in the memory by running or executing computer instructions, computer programs, code sets or instruction sets stored in the memory, and performs various functions and processes data of the present application. The processor can be at least one of an application specific integrated circuit, a digital signal processor, a digital signal processing device, a programmable logic device, a field programmable gate array, a central processing unit, a controller, a microcontroller and a microprocessor. It can be understood that for different devices, the electronic devices used to implement the functions of the above processor can also be other electronic devices. The memory can be used to store computer instructions, computer programs, code sets or instruction sets. The memory can include a program storage area and a data storage area, wherein the program storage area can store instructions for implementing an operating system, instructions for at least one function, and programs for implementing the above large visual language model vulnerability detection method based on security sensitive layer activation guidance, etc.; the data storage area can store data involved in the above large visual language model vulnerability detection method based on security sensitive layer activation guidance, etc.

[0160] In the embodiments of the present application, a computer readable storage medium is also provided, and the storage medium is used to store a computer program, and the computer program is used to make a computer execute the above large visual language model vulnerability detection method based on security sensitive layer activation guidance.

[0161] The above only describes the preferred embodiments of the present application, and it should be noted that for those skilled in the art, without departing from the principles of the present application, a number of improvements and refinements can be made, and these improvements and refinements should also be considered as the protection scope of the present application.

Claims

1. A large visual language model vulnerability detection method based on security sensitive layer activation guidance, characterized in that, The method comprises the following steps: S1, constructing a security-sensitive layer activation guide sample data set, selecting a method for covering large visual language model multi-modal attack scenarios to generate text and image inputs; S2, by quantitatively analyzing the activation difference of the intermediate layer of the large visual language model under attack and normal input, the security-sensitive layer combination is identified; S3, based on the identified security-sensitive layer combination and the output layer of the large visual language model, input any picture as the initial adversarial image, jointly apply the multiple losses of harmful content generation and rejection response suppression, and generate the process adversarial image with attack attributes through the optimization of the loss function; S4, further screening the security-sensitive layer combination in S2, recording the optimal security-sensitive layer combination and the harmful response path guided thereby, constructing a structured representation of potential security vulnerabilities of the large visual language model and generating the final adversarial image.

2. The method of claim 1, wherein the method is based on a security-sensitive layer activation-guided large visual language model vulnerability detection. In the S1, the method for covering large visual language model multi-modal attack scenarios includes a text-based adversarial perturbation attack method GCG, a visual-based adversarial perturbation attack method VAJM, a visual and text-based joint adversarial attack method UMK, and a visual prompt injection attack method Figstep, and the specific construction method of S1 is: a set of sample pairs of input-output pairs containing "harmful" and "harmless" semantics as sensitive layer activation guides, wherein, denotes a set of input-output pairs successfully guiding a large visual language model to output harmful responses, containing an optimized adversarial image and jailbreak text wherein, consists of a harmful query and a jailbreak suffix ; denotes a set of image-text inputs that the model fails to be successfully jailbroken, keeping safe responses under harmful queries , denotes benign image inputs, denotes the total amount of sample pairs in the set of sample pairs.

3. The method of claim 2, wherein the method is based on activation guidance of security-sensitive layers of a large visual language model. The S2 specifically comprises: S2.1, input the activation guide samples generated in S1 into the large visual language model to be tested, extract the hidden state representation of each layer, and perform difference quantification analysis on the activation results of each pair of activation guide samples in each layer of the model. The Euclidean distance is used to measure the activation distribution difference between harmful and harmless responses, and the safety sensitivity score of each layer is calculated , as shown in the following formula: ； wherein, , ] represents the activation guide sample pair constructed in S1, represents the hidden activation of the model in the first layer on the first sample pair, represents the total amount of input guide sample pairs, represents the security sensitivity score of the first layer, and finally calculates the average difference for each layer as its security sensitivity score, so as to realize the quantitative representation of the security sensitivity of each layer. S2.2, select security-sensitive layers and define their weights: ； wherein, represents the selection of the top layers with the highest safety sensitivity score from the first to the Lth layer of the model to form the safety sensitive layer combination ; represents the direct use of this score as the weight of the layer, the layer weight coefficient is defined here as the difference score of the activation of the layer under harmful and harmless input.

4. The method of claim 3, wherein the method is based on a security-sensitive layer activation-guided large visual language model vulnerability detection. The S3 specifically comprises the following steps: S3.1, hierarchical harmful content generation loss construction: define a harmful sample corpus H: ； Wherein, , represents the harmful statement of the target output, represents the number of harmful statements, the attack target is to maximize the generation probability of harmful statements in the security sensitive layer; the harmful content generation loss function is constructed as : ； ； wherein, represents the first layer model output head probability distribution under given conditions, represents the adversarial image to be optimized, whose pixel values are constrained within the standard image range; S3.2, hierarchical rejection response suppression mechanism design: to enhance the suppression of rejection mode in model response, construct a rejection response prefix set R: ； wherein, denotes a set of predefined typical rejection phrases, denotes a typical rejection phrase, denotes the total number of rejection phrases in the set; the constructed negative target is to minimize the conditional generation probability of the rejection response: ； ； wherein denotes the rejection response inhibition loss; S3.3, multi-objective joint optimization loss definition: ； wherein, denotes the harmful content generation loss, denotes the rejection response suppression loss, λ ∈ (0, 1) is a balance coefficient controlling the importance of the two losses, denotes the final multi-objective joint loss, by minimizing this total loss, both the two objectives of "encouraging harmful generation" and "suppressing rejection response" can be optimized simultaneously; S3.4, process adversarial image generation: based on the security-sensitive layer combination obtained in S2 and the multi-objective function constructed in S3.1-S3.3, input any picture as the initial adversarial image, and combine the gradient-based pixel update algorithm to optimize the image input, thereby generating a process adversarial image that can produce jailbreaking behavior.

5. The method of claim 1, wherein the method is based on a security-sensitive layer activation-guided large visual language model vulnerability detection. The S4 comprises the following steps: S4.1, combine the security sensitive layers obtained in S2 into a single security sensitive layer of size S4.2, encode the activation control of each layer using a binary bit. Each layer is considered as a search space, and the activation control of each layer is encoded using a binary bit. ； wherein, represents that the layer is attacked, represents that the layer does not participate in the attack optimization this time;​ S4.2, for all possible layer combination strategies , respectively, jailbreak attack experiments are conducted, and the corresponding attack success rates of the generated respective process-resistant images are recorded The ultimate goal is to find the optimal layer combination strategy so that the ability to generate harmful content is maximized, that is: ； This search process adopts an exhaustive strategy; S4.3, Once the optimal safety sensitive layer combination is determined The optimal path can be constrained accordingly, and only the corresponding layer is activated for intervention; specifically, the weight vector is updated as follows: ； wherein, represents the weight coefficient of the original of the security-sensitive layer; S4.4, after the directional constraint update of the weight vector, only gradient perturbation is performed on the optimal security sensitive layer to generate the final adversarial image that induces the large-scale visual language model to produce jailbreaking behavior .

6. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, The processor executes the computer program to realize the steps of the large visual language model vulnerability detection method based on security-sensitive layer activation guidance as claimed in any one of claims 1 to 5.

7. A computer-readable storage medium storing a computer program, wherein the computer program comprises the following steps of: receiving a request for a resource from a client; determining whether the client is authorized to access the resource; and if the client is authorized to access the resource, providing the resource to the client. The computer program is executed by the processor to realize the steps of the large visual language model vulnerability detection method based on security-sensitive layer activation guidance as claimed in any one of claims 1 to 5.

Images