A method for improving the security of multimodal large models with modal decoupling
By employing modal decoupling and multimodal context distillation, risk sources in large multimodal models are identified and optimized, addressing both insufficient and excessive security issues and improving the model's security and generalization ability.
Patent Information
- Application Number
- CN202411424470.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-10-12
- Publication Date
- 2025-11-14
- Estimated Expiration
- 2044-10-12
AI Technical Summary
Multimodal large models suffer from both insufficient and excessive security in terms of safety. Existing research cannot effectively capture complex risks, leading to models outputting harmful content or rejecting safe content.
Risk sources are identified through modal decoupling, and model safety is improved by combining multimodal context distillation and supervised fine-tuning with direct preference optimization. This includes steps such as risk detection, context distillation, supervised fine-tuning, and rejection sampling.
It improves the robustness of multimodal large models to harmful inputs, alleviates over-safety phenomena, and enhances the model's generalization ability and sample quality.
Smart Images
Figure CN119337874B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to a method for optimizing the security of multimodal large language models, specifically a method for improving the security of multimodal large language models with modal decoupling. Background Technology
[0002] Multimodal large models have garnered significant attention due to their superior performance in vision-language tasks. These models not only inherit the fundamental capabilities of large language models but also demonstrate broader application potential by incorporating visual information, such as image captioning, content creation, and medical diagnosis. However, as the application scope of multimodal large models continues to expand, concerns about their security and social impact are also increasing.
[0003] Existing research indicates that large language models face two extreme challenges in terms of security: on the one hand, there is "insufficient security," meaning that models not fine-tuned with security-related data have serious security vulnerabilities and are prone to outputting harmful content; on the other hand, there is "over-security," meaning that due to over-optimization of security, the model rejects prompts that appear similar to harmful content but are actually safe. These two problems are more pronounced in multimodal large models, and the complexity of the risks increases significantly after the introduction of visual modalities.
[0004] While some research has explored the security of large multimodal models, most approaches only offer a cursory analysis of risk sources, assuming the model can autonomously understand the corresponding risks based on data. This approach leads to an insufficient understanding of the complex risks in multimodal scenarios and an inability to effectively capture potential security vulnerabilities. Furthermore, it is prone to relying on superficial patterns, resulting in over-security. Summary of the Invention
[0005] To address the problems and needs existing in the background technology, this invention proposes a method for improving the security of multimodal large models based on modal decoupling. This method aims to enhance the security of multimodal large models (MLLMs) and effectively avoid over-security. First, this invention identifies and analyzes potential risk sources in the multimodal inputs through a meticulous risk deconstruction method. Based on this, synthetic data is obtained through multimodal context distillation. Finally, the model's security is optimized using methods such as supervised fine-tuning (SFT) and direct preference optimization (DPO), thereby improving the model's security.
[0006] The technical solution of the present invention is as follows:
[0007] I. A method for improving the security of multimodal large models with modal decoupling
[0008] S1: Based on the risk detection prompts, perform risk detection on each original input to obtain the corresponding risk set;
[0009] S2: Using each risk set as the context for context distillation prompts, and then performing context distillation on each original input to obtain the corresponding safety response, the training dataset is composed of all the original inputs and the corresponding safety responses; then, the multimodal large model is supervised fine-tuned using the training dataset to obtain the supervised fine-tuned multimodal large model;
[0010] S3: After rejecting sampling on the supervised fine-tuned multimodal large model, positive and negative sample pairs are obtained; based on the positive and negative sample pairs, the parameters of the supervised fine-tuned multimodal large model are fine-tuned using the direct preference optimization method to obtain a safe multimodal large model, thus improving the safety of the multimodal large model.
[0011] In S1, each original input is either image-only input, text-only input, or a combination of image and text input.
[0012] In step S1, the specific steps for risk detection for each mixed text and image input are as follows:
[0013] First, the modalities in each mixed image and text input are decoupled by modal decoupling to obtain image-only splitting results and text-only splitting results;
[0014] Based on the risk detection prompts, risk detection is then performed on the current mixed image and text input and the corresponding image-only and text-only splitting results to obtain the corresponding risk elements.
[0015] Then, after deduplicating the risk elements detected three times for the current mixed text and image input, the risk set for the current mixed text and image input is obtained.
[0016] The risk elements include the risk content, the risk source, and the risk category.
[0017] After deduplicating the risk elements detected three times from the current mixed text and image input, the risk set of the current mixed text and image input is obtained, including:
[0018] By using N-grammar to deduplicate the risk elements detected three times in the current mixed text and image input, the risk set of the current mixed text and image input is obtained.
[0019] In S3, after rejecting sampling of the supervised fine-tuned multimodal large model, positive and negative sample pairs are obtained, including:
[0020] First, based on the supervised fine-tuned multimodal large model M1, after sampling one original input in the training dataset, multiple corresponding response samples of different qualities are obtained;
[0021] Then, based on the prompt words used to judge the overall response quality of the model, the response quality coefficient is calculated for each of the obtained response samples to obtain the response quality coefficient corresponding to each response sample;
[0022] Next, based on the prompts indicating whether the model responds to whether it is aware of the existing risk observation results, the risk observation coefficients are calculated for all the response samples to obtain the risk observation coefficients corresponding to each response sample.
[0023] Finally, the response quality coefficient and risk observation coefficient of each response sample are added together to obtain the final sample sampling coefficient. The response samples with the largest and smallest sample sampling coefficients among all response samples are taken as positive and negative sample pairs.
[0024] Iterate through and process other original inputs in the training dataset to obtain positive and negative sample pairs corresponding to other original inputs, thereby obtaining all positive and negative sample pairs.
[0025] The context distillation prompts are prompts output by the constraint model, satisfying the following constraints:
[0026] The multimodal large model is required to self-check whether the risk actually exists, and it is also required that the multimodal large model fully understands the risk in the input and that the logic is fluent.
[0027] II. A Multimodal Large Model Security Enhancement System with Modal Decoupling
[0028] The risk set generation unit is used to perform risk detection on each original input based on risk detection prompts to obtain a risk set;
[0029] The training dataset construction unit is used to take each risk set as the context for context distillation prompts, and then perform context distillation on each original input to obtain the corresponding safety response, thereby obtaining the training dataset.
[0030] The supervised fine-tuning unit is used to perform supervised fine-tuning on a multimodal large model using the training dataset to obtain a supervised fine-tuned multimodal large model.
[0031] The model safety enhancement unit is used to reject sampling of the supervised fine-tuned multimodal large model to obtain positive and negative sample pairs; based on the positive and negative sample pairs, the direct preference optimization method is used to fine-tune the parameters of the supervised fine-tuned multimodal large model to obtain a safe multimodal large model.
[0032] III. A computer device
[0033] The computer device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the steps of the method.
[0034] IV. A computer-readable storage medium
[0035] A computer program is stored on a computer-readable storage medium, which, when executed by a processor, implements the steps of the method.
[0036] The beneficial effects of this invention are:
[0037] This invention proposes a method to enhance the security of multimodal large models with modal decoupling. This method can improve the model's robustness to harmful inputs while mitigating over-security.
[0038] This invention proposes an automated safety optimization framework. Models trained based on this framework have stronger generalization ability compared to direct distillation closed-source multimodal large models.
[0039] In the rejection sampling part, this invention proposes a scoring strategy based on risk observation facts to improve the sample quality of the model when performing preference sampling. Attached Figure Description
[0040] Figure 1 This is an overall flowchart of a method for improving the security of a multimodal large model through modal decoupling, according to an embodiment of the present invention. Detailed Implementation
[0041] The technical solution of the present invention will now be clearly and completely described with reference to the accompanying drawings. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention.
[0042] To make the objectives, technical solutions, and advantages of the present invention clearer, the embodiments of the present invention will be described in detail below with reference to the accompanying drawings.
[0043] like Figure 1 As shown, embodiments of the present invention include the following processes:
[0044] S1: Based on risk detection prompts, risk detection is performed on each original input to obtain the corresponding risk set. Each original input can be an image-only input, a text-only input, or a mixed image-text input, representing three different modalities. The training dataset contains inputs of all three modalities. Risk detection prompts are designed for image-only, text-only, and mixed image-text inputs, respectively. These prompts are used for risk detection, specifically by summarizing the safe usage strategies of the large language model to form risk category definitions; then, based on the differences between modalities and the risk category definitions, they are designed accordingly.
[0045] For image-only or text-only input, a risk detection is performed based on the risk detection prompts to obtain the corresponding risk factors and form a risk set.
[0046] For each mixed text and image input, the specific steps for risk detection are as follows:
[0047] First, the modalities in each mixed image and text input are decoupled by modal decoupling to obtain image-only splitting results and text-only splitting results;
[0048] Based on the risk detection prompts, risk detection is then performed on the current mixed text and image input, as well as the corresponding image-only and text-only splitting results, to obtain the corresponding risk elements. Each risk element includes the risk content, risk source, and risk category, as shown in the following formula:
[0049] O′=T(P x (i,t))∪T(P i (i))∪T(P t (t))
[0050] Among them, P i (i) represents the risk detection prompt word corresponding to the image input, P t (t) represents the risk detection prompt word corresponding to the text input, P x (i,t) represents the risk warning words corresponding to the mixed text and image input, O′ represents the risk set, and T() represents the multimodal large model used for risk detection.
[0051] Then, by deduplicating the risk elements detected three times from the current mixed text and image input using N-gram, the risk set of the current mixed text and image input is obtained, as shown in the following formula:
[0052] O = unique(O′)
[0053] Where O represents the set of risks after deduplication, and unique() represents the N-gram deduplication syntax.
[0054] S2: Using each risk set as the context of the context distillation prompt, and then performing context distillation on each original input to obtain the corresponding safety response, the training dataset is composed of all the original inputs and the corresponding safety responses; then, the multimodal large model is subjected to supervised fine-tuning (SFT) using the training dataset to obtain the supervised fine-tuned multimodal large model.
[0055] The context distillation cue words are the cue words output by the constraint model, satisfying the following constraints:
[0056] The multimodal large model is required to self-check whether the risk actually exists, and it is also required that the multimodal large model fully understands the risk in the input and that the logic is fluent.
[0057] During the supervised fine-tuning process, the model parameters are updated by minimizing the negative log-likelihood of the samples in the training dataset, resulting in the supervised fine-tuned multimodal large model M1.
[0058] S3: After rejecting sampling on the supervised fine-tuned multimodal large model, positive and negative sample pairs are obtained; based on the positive and negative sample pairs, the parameters of the supervised fine-tuned multimodal large model are fine-tuned using the Direct Preference Optimization (DPO) method to obtain a safe multimodal large model, thus improving the safety of the multimodal large model.
[0059] Among them, after rejecting sampling on the supervised fine-tuned multimodal large model, positive and negative sample pairs are obtained, including:
[0060] First, based on the supervised fine-tuned multimodal large model M1, after sampling one original input in the training dataset, multiple corresponding response samples of different qualities are obtained;
[0061] Then, based on the prompt words used to judge the overall response quality of the model, the response quality coefficient is calculated for each of the obtained response samples to obtain the response quality coefficient corresponding to each response sample;
[0062] Next, based on the prompts indicating whether the model responds to whether it is aware of the existing risk observation results, the risk observation coefficients are calculated for all the response samples to obtain the risk observation coefficients corresponding to each response sample.
[0063] Finally, the response quality coefficient and risk observation coefficient of each response sample are added together to obtain the final sample sampling coefficient. The response samples with the largest and smallest sample sampling coefficients among all response samples are taken as positive and negative sample pairs.
[0064] Iterate through and process other original inputs in the training dataset to obtain positive and negative sample pairs corresponding to other original inputs, thereby obtaining all positive and negative sample pairs.
[0065] The method proposed in this invention is applicable to fields such as multimodal large language models. This invention possesses the ability to efficiently and securely optimize multimodal large language models, and the model trained based on this invention exhibits high security and versatility.
[0066] Furthermore, as shown in Tables 1 and 2, the multimodal large model fine-tuned by this invention achieves state-of-the-art performance compared to existing methods on the SIUO, FigStep, and VLGuard datasets. Comparing with the LLaVA1.5 multimodal large model, the Safe & Effective metric of this invention on the SIUO dataset is improved from 22.16 to 28.74 compared to VLGuard. On the FigStep-Benign subset, this invention achieves a significant improvement over the existing method VLGuard, reducing the over-safety metric from 100 to 4. Simultaneously, on the MME, MM-Vet, and MMBeen datasets measuring the general capabilities of multimodal large models, this invention achieves higher scores than the existing method VLGuard. This invention can be widely applied in scenarios such as intelligent customer service and large language model agents.
[0067] Table 1 shows the evaluation results of the embodiments of the present invention on the SIUO, FigStep, and VLGuard datasets.
[0068]
[0069] Table 2 shows the evaluation results of the embodiments of the present invention on the full VLGuard test set and the MM-Vet, MME, and MMBench datasets.
[0070]
[0071] This invention also proposes a multimodal large model security enhancement system with modal decoupling, comprising:
[0072] The risk set generation unit is used to perform risk detection on each original input based on risk detection prompts to obtain a risk set;
[0073] The training dataset construction unit is used to take each risk set as the context for context distillation prompts, and then perform context distillation on each original input to obtain the corresponding safety response, thereby obtaining the training dataset.
[0074] The supervised fine-tuning unit is used to perform supervised fine-tuning on a multimodal large model using the training dataset to obtain a supervised fine-tuned multimodal large model.
[0075] The model safety enhancement unit is used to reject sampling of the supervised fine-tuned multimodal large model to obtain positive and negative sample pairs; based on the positive and negative sample pairs, the direct preference optimization method is used to fine-tune the parameters of the supervised fine-tuned multimodal large model to obtain a safe multimodal large model.
[0076] The present invention also proposes a computer device, which includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the steps of a method for improving the security of a multimodal large model with modal decoupling.
[0077] The present invention also proposes a computer-readable storage medium storing a computer program, wherein when the computer program is executed by a processor, it implements the steps of a method for improving the security of a multimodal large model with modal decoupling.
[0078] Finally, it should be noted that the above embodiments and descriptions are only used to illustrate the technical solutions of the present invention and not to limit it. Those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the disclosure of the technical solutions of the present invention, and all such modifications and substitutions should be covered within the protection scope of the claims of the present invention.
Claims
1. A method for improving the security of a multimodal large model with modal decoupling, characterized in that, Includes the following steps: S1: Based on the risk detection prompts, perform risk detection on each original input to obtain the corresponding risk set; each original input can be an image-only input, a text-only input, or a mixed image and text input; S2: Using each risk set as the context for context distillation prompts, and then performing context distillation on each original input to obtain the corresponding safety response, the training dataset is composed of all the original inputs and the corresponding safety responses; then, the multimodal large model is supervised fine-tuned using the training dataset to obtain the supervised fine-tuned multimodal large model; The context distillation prompts are prompts output by the constraint model, satisfying the following constraints: The multimodal large model is required to self-check whether the risk actually exists, and the multimodal large model is required to fully understand the risk in the input and have a smooth logic; S3: After rejecting sampling the supervised fine-tuned multimodal large model, positive and negative sample pairs are obtained; Based on positive and negative sample pairs, the direct preference optimization method is used to fine-tune the parameters of the supervised fine-tuned multimodal large model, resulting in a safe multimodal large model and improving the safety of the multimodal large model.
2. The method for improving the security of a multimodal large model with modal decoupling according to claim 1, characterized in that, In step S1, the specific steps for risk detection for each mixed text and image input are as follows: First, the modalities in each mixed image and text input are decoupled by modal decoupling to obtain image-only splitting results and text-only splitting results; Based on the risk detection prompts, risk detection is then performed on the current mixed image and text input and the corresponding image-only and text-only splitting results to obtain the corresponding risk elements. Then, after deduplicating the risk elements detected three times for the current mixed text and image input, the risk set for the current mixed text and image input is obtained.
3. The method for improving the security of a multimodal large model with modal decoupling according to claim 2, characterized in that, The risk elements include the risk content, the risk source, and the risk category.
4. The method for improving the security of a multimodal large model with modal decoupling according to claim 2, characterized in that, After deduplicating the risk elements detected three times from the current mixed text and image input, the risk set of the current mixed text and image input is obtained, including: By using N-grammar to deduplicate the risk elements detected three times in the current mixed text and image input, the risk set of the current mixed text and image input is obtained.
5. The method for improving the security of a multimodal large model with modal decoupling according to claim 1, characterized in that, In S3, after rejecting sampling of the supervised fine-tuned multimodal large model, positive and negative sample pairs are obtained, including: First, based on the supervised fine-tuned multimodal large model M1, after sampling one original input in the training dataset, multiple corresponding response samples of different qualities are obtained; Then, based on the prompt words used to judge the overall response quality of the model, the response quality coefficient is calculated for each of the obtained response samples to obtain the response quality coefficient corresponding to each response sample; Next, based on the prompts indicating whether the model responds to whether it is aware of the existing risk observation results, the risk observation coefficients are calculated for all the response samples to obtain the risk observation coefficients corresponding to each response sample. Finally, the response quality coefficient and risk observation coefficient of each response sample are added together to obtain the final sample sampling coefficient. The response samples with the largest and smallest sample sampling coefficients among all response samples are taken as positive and negative sample pairs. Iterate through and process other original inputs in the training dataset to obtain positive and negative sample pairs corresponding to other original inputs, thereby obtaining all positive and negative sample pairs.
6. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method for improving the security of a multimodal large model with modal decoupling as described in any one of claims 1 to 5.
7. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the steps of the method for improving the security of a multimodal large model with modal decoupling as described in any one of claims 1 to 5.
8. A computer program product comprising a computer program / instructions, characterized in that, When the computer program / instruction is executed by the processor, it implements the steps of the method for improving the security of a multimodal large model with modal decoupling as described in any one of claims 1 to 5.