Multi-modal large model jailbreaking attack defense method and system based on security prompt visual token reservation

Abstract

The present application relates to a kind of multi-modal large model jailbreak attack defense method and system based on security prompt visual token reservation, comprising: obtaining input image and user original text query;Obtain visual token sequence, based on the attention distribution of visual coding process, obtain semantic score;Input image and set security probe prompt word are jointly input into multi-modal model, and once no gradient forward propagation is executed, to obtain security prompt score;Combined with semantic score and security prompt score, obtain the joint score of each visual token, according to joint score from high to low order, select the highest score of the first K Visual token constructs reservation visual token subset, wherein, K It is reservation quantity;Based on the reservation visual token subset and user original text query, generate final response.Compared with prior art, the present application has the advantages of high defense reliability, security and normal visual understanding utility, etc.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence security technology, and in particular to a method and system for defending against multimodal large-scale jailbreak attacks based on the retention of visual tokens for security prompts. Background Technology

[0002] With the development of multimodal large models and visual language models, models can simultaneously receive image and text input and provide responses in tasks such as image understanding, image-text question answering, visual reasoning, OCR, and scene recognition. Multimodal capabilities enhance the practical value of models, but also expose them to combined image-text attacks. Attackers can combine harmful commands, misleading text, typographical perturbations, or risky images synthesized by generative models with user queries, weakening the model's original security alignment mechanisms and ultimately leading to the output of unsafe content.

[0003] Existing multimodal jailbreak defense methods typically address the issue using three approaches. The first approach converts images into text descriptions, attempting to shift the multimodal risk to a text security review problem. The second approach guides the model to reject non-compliant requests through prompts or security principles. The third approach identifies image tokens related to jailbreak or danger from intermediate layers of the model and prunes or suppresses these tokens. For example, patent application CN121834814A discloses a cross-modal attention-cooperative jailbreak attack method, which includes: constructing an input sequence representation, inputting it into a visual language model, and performing forward propagation; obtaining gradients with respect to the adversarial image representation and adversarial text suffixes based on the attention-guided loss cooperative function; optimizing the adversarial image representation and updating the adversarial text suffixes using gradient-guided pruning to optimize the attack target; iterating until the attention-guided cooperative loss function converges; outputting optimized unified multimodal knowledge; and generating an output response characterizing a successful jailbreak attack.

[0004] However, existing multimodal jailbreak defense methods still have the following shortcomings: First, directly converting an image into a text description will lose the fine-grained visual cues in the image and may introduce additional generation errors; when the attack image contains layout, local symbols, implicit risks or contextual cues, plain text descriptions are difficult to fully express these information.

[0005] Second, relying solely on defensive prompts or security principles usually increases input length and inference latency, and is easily covered or weakened by adversarial instructions; in the case of strong interference from image modalities, the model may still deviate from the safe rejection mode.

[0006] Third, existing hazard token-based pruning methods focus on deleting or suppressing salient risk regions in images. However, the inventors discovered that the visual evidence required to trigger a model's safe rejection is not the same as the hazard regions that should be deleted. Images also contain visual cues that help the model identify potential risks and activate safe judgments. Simply deleting salient risk regions may simultaneously destroy the contextual evidence upon which the model relies for safe judgments.

[0007] Fourth, existing methods rarely balance security, utility for normal visual tasks, and inference efficiency simultaneously. If too many visual tokens are pruned, the model's performance degrades in normal tasks such as OCR, fine-grained recognition, and person or scene identification; if no compression is performed or multiple generation steps are added, the defense overhead is high.

[0008] Therefore, there is an urgent need for a multimodal jailbreak attack defense method that can dynamically identify and retain key visual tokens that are helpful for security judgment during the inference stage, while maintaining the semantics of the main image and the normal utility of the model. Summary of the Invention

[0009] The purpose of this invention is to overcome the shortcomings of the existing technology and provide a multimodal large model jailbreak attack defense method and system based on the retention of security prompt visual tokens. It can dynamically identify and retain key visual tokens that help with security judgment during the inference stage, while maintaining the semantics of the main image and the normal utility of the model.

[0010] The objective of this invention can be achieved through the following technical solutions: A method for defending against multimodal large-scale jailbreak attacks based on the retention of visual tokens for security prompts includes the following steps: Obtain the input image and the user's original text query; The input image is visually encoded to obtain a sequence of visual tokens. Based on the attention distribution of the visual encoding process, the contribution of each visual token to the overall semantic representation of the image is calculated to obtain a semantic score. The input image and the set security detection prompt words are input into the multimodal model, and a gradient-free forward propagation is performed. The cross-modal attention of the text token corresponding to the security detection prompt word to each visual token is extracted and aggregated in the multi-head dimension and the text token dimension to obtain the security prompt score of each visual token. The semantic score and security hint score are combined to obtain a joint score for each visual token. The tokens are then sorted from highest to lowest joint score, and the highest-scoring tokens are selected. K Each visual token constructs a subset of visual tokens, where... K To retain the quantity; The final response is generated based on the reserved visual token subset and the user's original text query.

[0011] Furthermore, the input image is visually encoded using a visual encoder based on a multimodal model; For visual encoders that include explicit CLS tokens, the average attention from the CLS tokens to each visual token is used as the semantic score; for visual encoders that do not include explicit CLS tokens, the average input attention received by the visual tokens is used as the semantic score.

[0012] Furthermore, for visual encoders that include explicit CLS tokens, the first The semantic score of each visual token is represented as follows:

[0013] in, For semantic score, For the number of attention heads, Indicates the visual encoder's first... Layer In each attention focus, the CLS token points to the visual token. Attention weights; For visual encoders that do not contain explicit CLS tokens, the first The semantic score of each visual token is represented as follows:

[0014] in, Indicates the first The visual token points to the first... The attention weight of each visual token, where N represents the total number of visual tokens.

[0015] Furthermore, the cross-modal attention of the security detection prompt word corresponding to the text token pointing to the visual token is derived from the language model of the multimodal model. Layer extraction, the security prompt score is represented as:

[0016] in, To score points for safety tips, For the number of attention heads, This represents the set of text tokens corresponding to security detection prompts. Representation of the language model Layer Attention head Chinese text token Pointing to visual token Cross-modal attention weights.

[0017] Furthermore, before obtaining the joint score, the semantic score and the security prompt score are normalized respectively.

[0018] Furthermore, the normalization process employs Min-Max normalization, expressed as:

[0019]

[0020] in, and These are the normalized semantic score and the security suggestion score, respectively. , This is the original semantic score. , The original safety prompt score, To prevent division by zero of extremely small constants.

[0021] Furthermore, the joint score is expressed as:

[0022] in, β This is the risk warning intensity coefficient. For combined scores.

[0023] Furthermore, the number of reservations The calculation method is as follows:

[0024] in, ρ Preserve proportions for visual tokens. This indicates rounding up.

[0025] Furthermore, the method also includes: Based on the retained subset of visual tokens, delete the unselected visual tokens and simultaneously delete or update the corresponding items in the attention mask.

[0026] This invention also provides a multimodal large-scale jailbreak attack defense system based on security prompt visual token retention, comprising: Input acquisition module: used to acquire input images and user's original text query; Visual encoding module: used to perform visual encoding on the input image to obtain a visual token sequence; Semantic scoring module: Based on the attention distribution of the visual encoding process, it calculates the contribution of each visual token to the overall semantic representation of the image and obtains a semantic score; Security detection module: used to input the input image and the set security detection prompts into the multimodal model and perform a gradient-free forward propagation; Security scoring module: used to extract the cross-modal attention of the text token corresponding to the security detection prompt word to each visual token, and aggregate it in the multi-head dimension and the text token dimension to obtain the security prompt score of each visual token; Joint filtering module: Combines the semantic score and security hint score to obtain a joint score for each visual token, sorts them from highest to lowest joint score, and selects the top-scoring tokens. K Each visual token constructs a subset of visual tokens, where... K To retain the quantity; Security Response Output Module: Used to generate the final response based on the reserved visual token subset and the user's original text query.

[0027] Compared with the prior art, the present invention has the following beneficial effects: First, this invention calculates a semantic score and a security hint score for each visual token in the input image, thereby determining the subset of retained visual tokens used for the final response generation. This approach does not simply delete dangerous tokens, but rather identifies and retains security hint visual tokens that can help the model make security judgments. It leverages the model's own security alignment capabilities to transform visual evidence into valid grounds for triggering a security refusal.

[0028] Second, this invention jointly models the security prompt score and the semantic importance score, taking into account both security and normal visual understanding utility under a fixed token budget. This avoids the destruction of semantic information caused by focusing only on the security area, and also avoids insufficient security triggering caused by focusing only on the semantic area.

[0029] Third, this invention is executed dynamically during the inference phase, without the need to retrain or fine-tune the original multimodal large model, without changing the original model parameters, and can be combined with different visual language model architectures, thus having good deployment convenience.

[0030] Fourth, by deleting unselected visual tokens, this invention reduces the visual token load in subsequent cross-modal interactions and generation stages, thereby improving security while reducing inference latency and increasing throughput.

[0031] Fifth, this invention is compatible with visual encoders with and without explicit CLS tokens, and can adapt to different multimodal model structures such as LLaVA, ShareGPT4V, and Qwen2.5-VL.

[0032] Sixth, this invention can significantly reduce the success rate of jailbreak attacks such as MM-SafetyBench and JailbreakV-28K, and basically maintain the model's normal visual understanding ability in MME perception tasks. Attached Figure Description

[0033] Figure 1 This is a schematic flowchart of the method of the present invention; Figure 2 This is a schematic diagram of the system of the present invention. Detailed Implementation

[0034] The present invention will now be described in detail with reference to the accompanying drawings and specific embodiments. These embodiments are based on the technical solution of the present invention and provide detailed implementation methods and specific operating procedures. However, the scope of protection of the present invention is not limited to the following embodiments.

[0035] Terminology Explanation token: can also be expressed as "token", "visual token" or "image token".

[0036] Example 1 This embodiment provides a multimodal large-scale model jailbreak attack defense method based on the retention of security prompt visual tokens. This method leverages the model's existing security capabilities by first performing a gradient-free forward propagation using auxiliary security detection prompts to identify visual tokens helpful for security judgment from the cross-modal attention of the language model's intermediate layers. Simultaneously, it utilizes the internal attention of the visual encoder to identify visual tokens that significantly contribute to the semantics of the image subject. Then, it jointly scores the two types of scores and, under a fixed token budget, retains the subset of visual tokens with the highest scores, deleting unselected visual tokens and simultaneously updating the attention mask. Finally, it uses the retained visual tokens and the original user query input to generate a response from the multimodal model. This method, through joint analysis of semantic scores and security prompt scores, identifies and retains security prompt visual tokens that help the model make security judgments, effectively achieving multimodal jailbreak attack defense under limited defense overhead.

[0037] This method operates in a multimodal model that includes a visual encoder, a visual projection or multimodal alignment module, and a language model inference module, such as... Figure 1 As shown, it includes the following steps: Step S1: Obtain the input image Iand user's original text query x And obtain the set security detection prompt words. qs Setting parameters and the original multimodal model The settings include the visual token retention ratio. ρ Risk warning intensity coefficient β Language model layer for extracting cross-modal attention Visual coding layer .

[0038] The above security detection prompts qs The parameters can be dynamically configured according to different model architectures.

[0039] Specifically, the security detection prompt can be set to "Is this image safe? Describe any potential risks.", or it can be replaced with Chinese or other equivalent security risk detection prompts.

[0040] Language model layer, i.e., security prompt recognition layer The selection can be made through pre-experimentation or preset according to the model structure. Preferably, the selection method can include: comparing the differences between secure detection attention and native visual attention on harmful and harmless images, and selecting the layer with the larger difference as the preferred layer. .

[0041] Visual token retention ratio ρ The value can be set according to model performance, memory constraints, and security requirements, such as 0.2, 0.3, or 0.4.

[0042] Risk warning intensity β It can be a positive value to enhance the weight of the security prompt token in the filtering process; it can also be adjusted according to the application scenario.

[0043] Step S2: Visually encode the input image to obtain a visual token sequence.

[0044] Visual token sequence representation:

[0045] in, This represents the total number of visual tokens.

[0046] Step S3: Based on the attention distribution of the visual encoding process, calculate the contribution of each visual token to the overall semantic representation of the image, and obtain the semantic score.

[0047] The input image is visually encoded using a visual encoder. The attention distribution of the last layer or a specified layer of the visual encoder is extracted, and a semantic score is calculated. For visual encoders containing explicit CLS tokens, the average attention from the CLS tokens to each visual token is used as the semantic score; for visual encoders not containing explicit CLS tokens, the average input attention received by the visual tokens is used as the semantic score.

[0048] Specifically, for a visual encoder that includes an explicit CLS token, the first The semantic score of each visual token is represented as follows:

[0049] in, For semantic score, For the number of attention heads, Indicates the visual encoder's first... Layer In each attention focus, the CLS token points to the visual token. Attention weights; For visual encoders that do not contain explicit CLS tokens, the first The semantic score of each visual token is represented as follows:

[0050] in, Indicates the first The visual token points to the first... The attention weight of each visual token, where N represents the total number of visual tokens.

[0051] Step S4: Input the input image and the set security detection cue words into the multimodal model, perform one gradient-free forward propagation, extract the cross-modal attention from the text token corresponding to the security detection cue word to each visual token, and aggregate them in the multi-head dimension and the text token dimension to obtain the security cue score for each visual token. The higher the security cue score, the more likely the visual token contains visual evidence that helps the model identify potential risks and trigger a security refusal.

[0052] Specifically, the cross-modal attention between the text token corresponding to the security detection prompt and the visual token is derived from the language model of the multimodal model. Layer extraction, the security prompt score is represented as:

[0053] in, To score points for safety tips, For the number of attention heads, This represents the set of text tokens corresponding to security detection prompts. Representation of the language model Layer Attention head Chinese text token Pointing to visual token Cross-modal attention weights.

[0054] Step S5: Combine the semantic score and the security prompt score to obtain a joint score for each visual token.

[0055] Preferably, the semantic score and the security prompt score are first normalized to eliminate numerical scale differences caused by the different sources of the two types of scores. Specifically, the normalization process uses Min-Max normalization, expressed as:

[0056]

[0057] in, and These are the normalized semantic score and the security suggestion score, respectively. , This is the original semantic score. , The original safety prompt score, To prevent division by zero of extremely small constants.

[0058] Obtain the combined score for each visual token:

[0059] in, β This is the risk warning intensity coefficient. For combined scores.

[0060] Step S6: Sort the scores from highest to lowest according to the combined score, and select the top scorers based on the fixed budget. K Each visual token is used to construct a subset of visual tokens. ,in, K To retain the quantity.

[0061] Specifically, the token is stored as follows:

[0062] in, β This is the risk warning intensity coefficient. For combined scores, Indicates rounding up. To preserve the token index set.

[0063] Step S7: Process the visual token sequence based on the retained visual token subset, use a hard deletion strategy to delete unselected visual tokens, and simultaneously delete or update the corresponding items in the attention mask, while keeping the relative order of the tokens in the original sequence unchanged.

[0064] The purpose of synchronously deleting or updating the corresponding items in the attention mask is to ensure that after deleting unselected visual tokens, subsequent attention calculations by the model only apply to the retained visual tokens, preventing invalid positions of deleted tokens from continuing to participate in the calculation or causing sequence misalignment. This process maintains consistency between the visual token sequence, the attention mask, and the text-visual joint input sequence, preventing dimensionality mismatch and interference from invalid information, and improving the stability and efficiency of subsequent inference.

[0065] A subset of visual tokens will be retained. After passing through a visual projection or multimodal alignment module, the original multimodal model is input along with the original user query x to generate the final response. This includes responses such as refusal to answer, risk warnings, compliance alternatives, or normal visual Q&A results.

[0066] The final reasoning process can be represented as:

[0067] in, This indicates a visual projection or multimodal alignment module. This represents the original multimodal language model. This indicates that the model ultimately generates a response.

[0068] The above methods can be used to defend against jailbreak attacks under combined text and image input, as well as for scenarios such as image review assistance, multimodal security gateways, enterprise internal large model access control, and security enhancement of multimodal question-and-answer systems.

[0069] If the above methods are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0070] Example 2 refer to Figure 2 As shown, this embodiment provides a multimodal large-scale jailbreak attack defense system based on security prompt visual token retention, including: Input acquisition module: used to acquire input images and user's original text query; Visual encoding module: used to perform visual encoding on the input image to obtain a visual token sequence; Semantic scoring module: Based on the attention distribution of the visual encoding process, it calculates the contribution of each visual token to the overall semantic representation of the image and obtains a semantic score; Security detection module: used to input the input image and the set security detection prompts into the multimodal model and perform a gradient-free forward propagation; Security scoring module: used to extract the cross-modal attention of the text token corresponding to the security detection prompt word to each visual token, and aggregate it in the multi-head dimension and the text token dimension to obtain the security prompt score of each visual token; Joint filtering module: Combines the semantic score and security hint score to obtain a joint score for each visual token, sorts them from highest to lowest joint score, and selects the top-scoring tokens. K Each visual token constructs a subset of visual tokens, where... K To retain the quantity; Security Response Output Module: Used to generate the final response based on the reserved visual token subset and the user's original text query.

[0071] In another embodiment, the system also includes a parameter configuration module for configuring the security prompt recognition layer, visual token retention ratio, risk prompt strength, and security detection prompt words according to different model architectures.

[0072] The following application examples illustrate the specific implementation effects of the present invention under different evaluation dimensions. Each application example follows... Figure 1 The process is carried out according to the method shown, or by using... Figure 2 Each functional module in the system structure shown executes its corresponding steps.

[0073] In the test, LLaVA-1.5-7B, ShareGPT4V-7B and Qwen2.5-VL-7B were selected as the original multimodal models. For different models, the corresponding security prompt recognition layer Lc, visual token retention ratio ρ and risk prompt intensity β were set.

[0074] Specifically, LLaVA-1.5-7B uses Lc=30, ρ=0.3, and β=0.5; ShareGPT4V-7B uses Lc=6, ρ=0.2, and β=0.25; and Qwen2.5-VL-7B uses Lc=27, ρ=0.4, and β=0.25. These parameters are only used to illustrate the feasible configurations of the present invention and are not intended to limit the scope of protection of the present invention.

[0075] Application Example 1: Defense effectiveness of different models on jailbreak attack datasets This application example is used to verify the invention's ability to defend against combined text and image jailbreak attacks. During testing, an image containing potentially harmful and misleading content is input into the original multimodal model along with a user query. The system first performs a security probe forward propagation, extracting cross-modal attention from the text tokens corresponding to the security probe prompts to the visual tokens. It then combines the visual semantic importance score to complete the TopK visual token retention. Finally, the retained visual tokens and the original user query are fed into the model to generate the final response.

[0076] The average attack success rate was used as the evaluation metric. A lower attack success rate indicates that the model is less likely to be induced by jailbreak attacks to output unsafe content. Table 1 shows the comparison results of the three models on the MM-SafetyBench and JailbreakV-28K jailbreak attack datasets.

[0077] Table 1. Defense performance of different models on jailbreak attack datasets.

[0078] Application Example 2: Utility Preservation of Different Models on Normal Multimodal Tasks This application example verifies whether the invention maintains the model's normal visual understanding capabilities while enhancing security. During testing, the same token selection and inference process as in Application Example 1 was used, but the input content consisted of normal multimodal understanding task samples. The system only deleted visual tokens with low joint scores and retained those capable of expressing the subject's semantics or triggering necessary security judgments, thereby avoiding a significant decline in OCR, scene recognition, target attribute understanding, and visual question answering capabilities due to excessive pruning.

[0079] The total score of the MME perception task was used as the evaluation index; a higher total score indicates stronger normal multimodal perception capability. Table 2 shows a comparison of the scores of the three models in the undefended state and after adopting the present invention.

[0080] Table 2. Utility retention of different models on normal multimodal tasks. LLaVA-1.5-7B 1503.4 1492.6 Close to the original model ShareGPT4V-7B 1584.7 1567.6 Close to the original model Qwen2.5-VL-7B 1704.0 1714.7 Higher than the original model Application Example 3: Relative Results of Different Models in Inference Efficiency This application example is used to verify the computational efficiency of the present invention during the inference phase. During testing, the present invention adds a gradient-free security probe and token scoring process at the front end, but reduces the number of visual tokens in the subsequent main inference process, thereby reducing the computational load on visual tokens during cross-modal interaction and generation phases.

[0081] Based on the defenseless inference efficiency, the relative latency and relative throughput after adopting this invention were statistically analyzed. A relative latency of less than 1 indicates a reduction in inference latency, and a relative throughput of more than 1 indicates an improvement in processing capacity per unit time. Table 3 presents the relative results for the three models.

[0082] Table 3. Relative results of different models in terms of inference efficiency LLaVA-1.5-7B 0.97× 1.05× Reduced latency and increased throughput ShareGPT4V-7B 0.97× 1.04× Reduced latency and increased throughput Qwen2.5-VL-7B 0.93× 1.07× Reduced latency and increased throughput

[0083] As shown in Tables 1 to 3, this invention significantly reduces the success rate of jailbreak attacks across different models. In normal multimodal tasks, the MME perception score is close to the original model, with some models showing slight improvements. Regarding inference efficiency, all three models exhibit relative latency reduction and relative throughput improvement. These results demonstrate that this invention achieves a good balance between security defense, utility preservation, and inference efficiency.

[0084] The preferred embodiments of the present invention have been described in detail above. The present invention also includes technical solutions formed by any combination of the above-described technical features. It should be understood that those skilled in the art can make numerous modifications and variations based on the concept of the present invention without creative effort. Therefore, all technical solutions that can be obtained by those skilled in the art based on the concept of the present invention through logical analysis, reasoning, or limited experimentation on the basis of existing technology should be within the scope of protection defined by the claims.

Claims

1. A multi-modal large model jailbreaking attack defense method based on security prompt visual token reservation, characterized in that, Includes the following steps: Obtain the input image and the user's original text query; The input image is visually encoded to obtain a sequence of visual tokens. Based on the attention distribution of the visual encoding process, the contribution of each visual token to the overall semantic representation of the image is calculated to obtain a semantic score. The input image and the set security detection prompt words are input into the multimodal model, and a gradient-free forward propagation is performed. The cross-modal attention of the text token corresponding to the security detection prompt word to each visual token is extracted and aggregated in the multi-head dimension and the text token dimension to obtain the security prompt score of each visual token. A joint score of each visual token is obtained in combination with the semantic score and the security hint score, the visual tokens are ranked in descending order of the joint scores, and the top K visual tokens with the highest scores are selected to construct a reserved visual token subset, wherein, K is the reserved number. The final response is generated based on the reserved visual token subset and the user's original text query.

2. The multi-modal large model jailbreaking attack defense method based on security prompt visual token reservation according to claim 1, characterized in that, The input image is visually encoded using a visual encoder based on a multimodal model. For visual encoders that include explicit CLS tokens, the average attention from the CLS tokens to each visual token is used as the semantic score; for visual encoders that do not include explicit CLS tokens, the average input attention received by the visual tokens is used as the semantic score.

3. The multi-modal large model jailbreaking attack defense method based on security prompt visual token reservation according to claim 2, characterized in that, For visual encoders that include an explicit CLS token, the semantic score for the i-th visual token is represented as: ​ wherein, is the semantic score, is the number of attention heads, denotes the attention weight of the CLS token in the layer and the attention head to the visual token ; For visual encoders that do not include an explicit CLS token, the semantic score for the jth visual token is represented as: ​ in, Indicates the first The visual token points to the first... The attention weight of each visual token, where N represents the total number of visual tokens.

4. The multi-modal large model jailbreaking attack defense method based on security prompt visual token reservation according to claim 1, characterized in that, The security detection prompt word corresponds to the text token pointing to the visual token. The cross-modal attention is drawn from the language model of the multimodal model. Layer extraction, the security prompt score is represented as: in, To score points for safety tips, For the number of attention heads, This represents the set of text tokens corresponding to security detection prompts. Representation of the language model Layer Attention head Chinese text token Pointing to visual token Cross-modal attention weights.

5. The multi-modal large model jailbreaking attack defense method based on security prompt visual token reservation according to claim 1, characterized in that, Before obtaining the joint score, the semantic score and the security prompt score are normalized respectively.

6. The multi-modal large model jailbreaking attack defense method based on security prompt visual token reservation according to claim 5, characterized in that, The normalization process uses Min-Max normalization, expressed as: wherein, and are normalized semantic score and safety cue score, respectively, , is the original semantic score, , is the original safety cue score, is a small constant to prevent division by zero.

7. The multi-modal large model jailbreaking attack defense method based on security prompt visual token reservation according to claim 6, characterized in that, The joint score is expressed as: wherein, β is a risk alert intensity coefficient, is a joint score.

8. The method of claim 1, wherein the method is characterized by: the reserved number is calculated as follows: in, ρ Preserve proportions for visual tokens. This indicates rounding up.

9. The multimodal large-scale jailbreak attack defense method based on security prompt visual token retention according to claim 1, characterized in that, The method also includes: Based on the retained subset of visual tokens, delete the unselected visual tokens and simultaneously delete or update the corresponding items in the attention mask.

10. A multi-modal large model jailbreaking attack defense system based on security prompt visual token reservation, characterized in that, include: Input acquisition module: used to acquire input images and user's original text query; Visual encoding module: used to perform visual encoding on the input image to obtain a visual token sequence; Semantic scoring module: Based on the attention distribution of the visual encoding process, it calculates the contribution of each visual token to the overall semantic representation of the image and obtains a semantic score; Security detection module: used to input the input image and the set security detection prompts into the multimodal model and perform a gradient-free forward propagation; Security scoring module: used to extract the cross-modal attention of the text token corresponding to the security detection prompt word to each visual token, and aggregate it in the multi-head dimension and the text token dimension to obtain the security prompt score of each visual token; The joint screening module combines the semantic score and the security prompt score to obtain a joint score of each visual token, sorts the visual tokens in descending order of the joint score, selects the top K visual tokens with the highest scores to construct a reserved visual token subset, wherein, K is the reserved number. Security Response Output Module: Used to generate the final response based on the reserved visual token subset and the user's original text query.

Images