Method for improving security of modal decoupling multi-modal large model

Abstract

The invention discloses a method for improving the safety of a multi-modal large model based on modal decoupling. The method comprises the following steps: firstly, performing risk detection on each original input according to a risk detection prompt word to obtain a corresponding risk set; then, taking each risk set as a context of a context distillation cue word, and performing context distillation on each original input to obtain a corresponding security response, thereby obtaining a training data set; performing supervised fine tuning on the multi-modal large model to obtain a model after supervised fine tuning; then, after sampling rejection is carried out on the multi-modal large model after supervised fine tuning, positive and negative sample pairs are obtained; and finally, performing parameter fine tuning on the multi-modal large model after supervised fine tuning by using a direct preference optimization method to obtain a safe multi-modal large model. According to the method, the security optimization of the multi-modal large model is realized, the robustness of the multi-modal large model for risk input is remarkably improved, and the universality of the model is kept.

Description

Technical Field

[0001] The present invention relates to a method for optimizing the security of a multimodal large language model in the field of multimodal large language models, and specifically relates to a method for improving the security of a multimodal large language model with respect to modal decoupling. Background Art

[0002] Large multimodal models have attracted much attention due to their excellent performance in vision-language tasks. These models not only inherit the basic capabilities of large language models, but also show potential for a wider range of applications by combining visual information, such as image description, content creation, and medical diagnosis. However, as the scope of application of large multimodal models continues to expand, concerns about their safety and social impact are also increasing.

[0003] Existing research shows that large language models face two extreme challenges in terms of security: on the one hand, there is "insufficient security", that is, models that have not been fine-tuned with security-related data have serious security vulnerabilities and are prone to output harmful content; on the other hand, there is "over-security", that is, due to the excessive optimization of the model in terms of security, it rejects those prompts that are superficially similar to harmful content but are actually safe. These two problems are more serious in large multimodal models, and the complexity of the risks increases significantly after the introduction of the visual modality.

[0004] Although some studies have explored the security of large multimodal models, most solutions only conduct a rough analysis of the risk sources, assuming that the model can autonomously understand the corresponding risks based on the data. This approach leads to insufficient understanding of complex risks in multimodal scenarios and inability to effectively capture potential safety hazards. At the same time, it is easy to rely on some superficial patterns, which leads to excessive security issues. Summary of the invention

[0005] In order to solve the problems and needs existing in the background technology, the present invention proposes a multimodal large model security improvement method based on modal decoupling, which aims to enhance the security of multimodal large models (MLLM) and effectively avoid the problem of over-security. The present invention first identifies and analyzes the potential risk sources in multimodal inputs through a detailed risk deconstruction method, and on this basis obtains synthetic data through multimodal context distillation. Finally, the model is optimized for security through methods such as supervised fine-tuning (SFT) and direct preference optimization (DPO) to improve the security of the model.

[0006] The technical solution of the present invention is as follows:

[0007] 1. A method for improving the safety of multi-modal large models with modal decoupling

[0008] S1: After performing risk detection on each original input according to the risk detection prompt word, the corresponding risk set is obtained;

[0009] S2: Each risk set is used as the context of the context distillation prompt word, and then each original input is context distilled to obtain the corresponding safety response. All original inputs and corresponding safety responses form a training data set. Then, the multimodal large model is fine-tuned in a supervised manner using the training data set to obtain a multimodal large model after supervised fine-tuning.

[0010] S3: After rejection sampling, the supervised fine-tuned multimodal large model is subjected to positive and negative sample pairs. Based on the positive and negative sample pairs, the direct preference optimization method is used to fine-tune the parameters of the supervised fine-tuned multimodal large model to obtain a secure multimodal large model, thus improving the security of the multimodal large model.

[0011] In S1, each original input is an image-only input, a text-only input, or an image-text mixed input.

[0012] In S1, for each mixed image and text input, the specific steps of risk detection are as follows:

[0013] First, the modality in each image-text mixed input is split through modality decoupling to obtain image-only splitting results and text-only splitting results.

[0014] Then, according to the risk detection prompt words, risk detection is performed on the current image-text mixed input and the corresponding image-only segmentation results and text-only segmentation results to obtain the corresponding risk elements;

[0015] Then, after removing duplicates of the risk elements detected three times in the current image-text mixed input, a risk set of the current image-text mixed input is obtained.

[0016] The risk elements include risk content, risk source and risk category.

[0017] After deduplication of the risk elements detected three times in the current mixed image and text input, a risk set of the current mixed image and text input is obtained, including:

[0018] The risk elements detected three times in the current mixed text and image input are deduplicated using N-grams to obtain a risk set for the current mixed text and image input.

[0019] In S3, after rejection sampling is performed on the supervised fine-tuned multimodal large model, positive and negative sample pairs are obtained, including:

[0020] First, based on the supervised fine-tuned multimodal large model M1, an original input in the training dataset is sampled to obtain multiple corresponding response samples of different qualities;

[0021] Then, according to the prompt words for judging the overall response quality of the model, the response quality coefficients of all the response samples obtained are calculated respectively to obtain the response quality coefficients corresponding to each response sample;

[0022] Next, based on the prompt words used to determine whether the model response is aware of the existing risk observation results, the risk observation coefficients are calculated for all the response samples obtained to obtain the risk observation coefficients corresponding to each response sample;

[0023] Finally, the final sample sampling coefficient is obtained by adding the response quality coefficient and risk observation coefficient of each response sample, and the response samples with the largest and smallest sample sampling coefficients among all response samples are taken as positive and negative sample pairs;

[0024] Traverse and process other original inputs in the training data set to obtain positive and negative sample pairs corresponding to other original inputs, thereby obtaining all positive and negative sample pairs.

[0025] The context distillation prompt words are prompt words output by the constraint model and satisfy the following constraints:

[0026] The multimodal big model is required to self-check whether the risks really exist, and is required to fully understand the risks in the input and have smooth logic.

[0027] 2. A multi-modal large model safety improvement system with modal decoupling

[0028] A risk set generating unit, used to perform risk detection on each original input according to the risk detection prompt word to obtain a risk set;

[0029] The training data set construction unit is used to use each risk set as the context of the context distillation prompt word, and then obtain the corresponding safety response after context distillation of each original input, so as to obtain the training data set.

[0030] A supervised fine-tuning unit is used to perform supervised fine-tuning on the multimodal large model using the training data set to obtain a supervised fine-tuned multimodal large model;

[0031] The model security enhancement unit is used to perform rejection sampling on the supervised fine-tuned multimodal large model to obtain positive and negative sample pairs; based on the positive and negative sample pairs, the direct preference optimization method is used to fine-tune the parameters of the supervised fine-tuned multimodal large model to obtain a secure multimodal large model.

[0032] 3. A computer device

[0033] The computer device comprises a memory and a processor, wherein the memory stores a computer program, and the steps of the method are implemented when the processor executes the computer program.

[0034] 4. A computer-readable storage medium

[0035] A computer program is stored on the computer-readable storage medium, and when the computer program is executed by a processor, the steps of the method described are implemented.

[0036] The beneficial effects of the present invention are:

[0037] The present invention proposes a multi-modal large model security improvement method with modal decoupling, which can improve the robustness of the model in the face of harmful inputs and alleviate the over-security phenomenon.

[0038] The present invention proposes an automated security optimization framework. The model trained based on this framework has stronger generalization ability than the directly distilled closed-source multimodal large model.

[0039] In the rejection sampling part, the present invention proposes a scoring strategy based on risk observation facts to improve the sample quality of the model during preference sampling. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 It is an overall flow chart of a method for improving the security of a multi-modal large model with modal decoupling according to an embodiment of the present invention. DETAILED DESCRIPTION

[0041] The technical solution of the present invention will be described clearly and completely below in conjunction with the accompanying drawings of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in the field without creative work are within the scope of protection of the present invention.

[0042] In order to make the objectives, technical solutions and advantages of the present invention more clearly understood, embodiments of the present invention will be described in detail with reference to the accompanying drawings.

[0043] like Figure 1 As shown, the embodiment of the present invention includes the following processes:

[0044] S1: After performing risk detection on each original input according to the risk detection prompt words, the corresponding risk set is obtained; each original input is image-only input, text-only input, or image-text mixed input, that is, input of three different modalities. The training data set contains inputs of three modalities. Corresponding risk detection prompt words are designed for image-only input, text-only input, and image-text mixed input respectively. Risk detection prompt words are used for risk detection, specifically by combing the safe use strategy of the large language model to form a risk category definition; then they are designed based on the differences between the modalities and the risk category definition.

[0045] For image input only or text input only, a risk detection is performed according to the risk detection prompt words to obtain the corresponding risk factors and form a risk set.

[0046] For each mixed image and text input, the specific steps of risk detection are as follows:

[0047] First, the modality in each image-text mixed input is split through modality decoupling to obtain image-only splitting results and text-only splitting results.

[0048] Then, according to the risk detection prompt words, risk detection is performed on the current image-text mixed input and the corresponding image-only segmentation results and text-only segmentation results to obtain the corresponding risk elements; the risk elements include risk content, risk source and risk category, and the formula is as follows:

[0049] O′=T(P x (i,t))∪T(P i (i))∪T(P t (t))

[0050] Among them, P i (i) represents the risk detection prompt word corresponding to the image input, P t (t) represents the risk detection prompt word corresponding to the text input, P x (i, t) represents the risk warning words corresponding to the mixed input of pictures and texts, O′ represents the risk set, and T() represents the multimodal large model for risk detection.

[0051] Then, after removing duplicates of the risk elements detected three times in the current mixed image and text input through N-gram, the risk set of the current mixed image and text input is obtained. The formula is as follows:

[0052] O = unique(O′)

[0053] Among them, O represents the risk set after deduplication, and unique() represents the N-ary deduplication syntax.

[0054] S2: Each risk set is used as the context of the context distillation prompt word, and then each original input is context distilled to obtain the corresponding safety response. The training data set is composed of all the original inputs and the corresponding safety responses. Then, the multimodal large model is supervised fine-tuned (SFT) using the training data set to obtain the supervised fine-tuned multimodal large model.

[0055] Among them, the context distillation prompt word is the prompt word output by the constraint model, which satisfies the following constraints:

[0056] The multimodal big model is required to self-check whether the risks really exist, and is required to fully understand the risks in the input and have smooth logic.

[0057] During the supervised fine-tuning of the model, the model parameters are updated by minimizing the negative log-likelihood of the samples in the training data set to obtain the supervised fine-tuned multimodal large model M1.

[0058] S3: After rejection sampling, the supervised fine-tuned multimodal large model is subjected to positive and negative sample pairs. Based on the positive and negative sample pairs, the direct preference optimization method (DPO) is used to fine-tune the parameters of the supervised fine-tuned multimodal large model to obtain a secure multimodal large model, thus improving the security of the multimodal large model.

[0059] Among them, after rejection sampling of the supervised fine-tuned multimodal large model, positive and negative sample pairs are obtained, including:

[0060] First, based on the supervised fine-tuned multimodal large model M1, an original input in the training dataset is sampled to obtain multiple corresponding response samples of different qualities;

[0061] Then, according to the prompt words for judging the overall response quality of the model, the response quality coefficients of all the response samples obtained are calculated respectively to obtain the response quality coefficients corresponding to each response sample;

[0062] Next, based on the prompt words used to determine whether the model response is aware of the existing risk observation results, the risk observation coefficients are calculated for all the response samples obtained to obtain the risk observation coefficients corresponding to each response sample;

[0063] Finally, the final sample sampling coefficient is obtained by adding the response quality coefficient and risk observation coefficient of each response sample, and the response samples with the largest and smallest sample sampling coefficients among all response samples are taken as positive and negative sample pairs;

[0064] Traverse and process other original inputs in the training data set to obtain positive and negative sample pairs corresponding to other original inputs, thereby obtaining all positive and negative sample pairs.

[0065] The method proposed in the present invention is applicable to the fields of multimodal large language models, etc. The present invention has the ability to efficiently optimize the security of multimodal large language models, and the model trained based on this invention has high security and general capabilities.

[0066] In addition, as shown in Tables 1 and 2, the multimodal large model fine-tuned by the present invention has achieved the most advanced performance on the SIUO, FigStep, and VLGuard datasets compared with existing methods. Compared with the LLaVA1.5 multimodal large model, the Safe & Effective index of the present invention on the SIUO dataset is improved from 22.16 to 28.74 compared with VLGuard; on the FigStep-Benign subset, the present invention has achieved significant improvement compared with the existing method VLGuard, and the over-safety index has been reduced from 100 to 4. At the same time, on the MME, MM-Vet, and MMBench datasets that measure the general capabilities of multimodal large models, the present invention has achieved higher scores than the existing method VLGuard. The present invention can be widely used in scenarios such as intelligent customer service and large language model agents (Agent).

[0067] Table 1 shows the evaluation results of the embodiments of the present invention on the SIUO, FigStep, and VLGuard datasets.

[0068]

[0069] Table 2 shows the evaluation results of the embodiment of the present invention on the VLGuard full test set and the MM-Vet, MME, and MMBench data sets.

[0070]

[0071] The present invention also proposes a modal decoupled multi-modal large model safety improvement system, comprising:

[0072] A risk set generating unit, used to perform risk detection on each original input according to the risk detection prompt word to obtain a risk set;

[0073] The training data set construction unit is used to use each risk set as the context of the context distillation prompt word, and then obtain the corresponding safety response after context distillation of each original input, so as to obtain the training data set.

[0074] A supervised fine-tuning unit is used to perform supervised fine-tuning on the multimodal large model using the training data set to obtain a supervised fine-tuned multimodal large model;

[0075] The model security enhancement unit is used to perform rejection sampling on the supervised fine-tuned multimodal large model to obtain positive and negative sample pairs; based on the positive and negative sample pairs, the direct preference optimization method is used to fine-tune the parameters of the supervised fine-tuned multimodal large model to obtain a secure multimodal large model.

[0076] The present invention also proposes a computer device, which includes a memory and a processor. The memory stores a computer program, and when the processor executes the computer program, it implements the steps of a method for improving the security of a multi-modal large model with modal decoupling.

[0077] The present invention also proposes a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of a method for improving the security of a multi-modal large model with modal decoupling are implemented.

[0078] Finally, it should be noted that the above embodiments and explanations are only used to illustrate the technical solution of the present invention rather than to limit it. Those skilled in the art should understand that the technical solution of the present invention can be modified or replaced by equivalents without departing from the spirit and scope disclosed in the technical solution of the present invention, which should be included in the scope of protection of the claims of the present invention.

Claims

1. A method for improving the safety of a multi-modal large model with modal decoupling, characterized in that: The following steps are involved: S1: After performing risk detection on each original input according to the risk detection prompt word, the corresponding risk set is obtained; S2: Each risk set is used as the context of the context distillation prompt word, and then each original input is context distilled to obtain the corresponding safety response. All original inputs and corresponding safety responses form a training data set. Then, the multimodal large model is fine-tuned in a supervised manner using the training data set to obtain a multimodal large model after supervised fine-tuning. S3: After rejection sampling, the supervised fine-tuned multimodal large model is subjected to positive and negative sample pairs. Based on the positive and negative sample pairs, the direct preference optimization method is used to fine-tune the parameters of the supervised fine-tuned multimodal large model to obtain a secure multimodal large model, thus improving the security of the multimodal large model.

2. According to the method for improving the safety of a multi-modal large model with modal decoupling according to claim 1, it is characterized in that: In S1, each original input is an image-only input, a text-only input, or an image-text mixed input.

3. The method for improving the safety of a multi-modal large model with modal decoupling according to claim 2 is characterized in that: In S1, for each mixed image and text input, the specific steps of risk detection are as follows: First, the modality in each image-text mixed input is split through modality decoupling to obtain image-only splitting results and text-only splitting results. Then, according to the risk detection prompt words, risk detection is performed on the current image-text mixed input and the corresponding image-only segmentation results and text-only segmentation results to obtain the corresponding risk elements; Then, after removing duplicates of the risk elements detected three times in the current image-text mixed input, a risk set of the current image-text mixed input is obtained.

4. The method for improving the safety of a multi-modal large model with modal decoupling according to claim 3 is characterized in that: The risk elements include risk content, risk source and risk category.

5. The method for improving the safety of a multi-modal large model with modal decoupling according to claim 3 is characterized in that: After deduplication of the risk elements detected three times in the current mixed image and text input, a risk set of the current mixed image and text input is obtained, including: The risk elements detected three times in the current mixed text and image input are deduplicated using N-grams to obtain a risk set for the current mixed text and image input.

6. The method for improving the safety of a multi-modal large model with modal decoupling according to claim 1 is characterized in that: In S3, after rejection sampling is performed on the supervised fine-tuned multimodal large model, positive and negative sample pairs are obtained, including: First, based on the supervised fine-tuned multimodal large model M1, an original input in the training dataset is sampled to obtain multiple corresponding response samples of different qualities; Then, according to the prompt words for judging the overall response quality of the model, the response quality coefficients of all the response samples obtained are calculated respectively to obtain the response quality coefficients corresponding to each response sample; Next, based on the prompt words used to determine whether the model response is aware of the existing risk observation results, the risk observation coefficients are calculated for all the response samples obtained to obtain the risk observation coefficients corresponding to each response sample; Finally, the final sample sampling coefficient is obtained by adding the response quality coefficient and risk observation coefficient of each response sample, and the response samples with the largest and smallest sample sampling coefficients among all response samples are taken as positive and negative sample pairs; Traverse and process other original inputs in the training data set to obtain positive and negative sample pairs corresponding to other original inputs, thereby obtaining all positive and negative sample pairs.

7. The method for improving the safety of a multi-modal large model with modal decoupling according to claim 1 is characterized in that: The context distillation prompt words are prompt words output by the constraint model and satisfy the following constraints: The multimodal big model is required to self-check whether the risks really exist, and is required to fully understand the risks in the input and have smooth logic.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of a method for improving the safety of a multi-modal large model with modal decoupling as described in any one of claims 1 to 7 are implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of a method for improving the safety of a multi-modal large model with modal decoupling as described in any one of claims 1 to 7 are implemented.

10. A computer program product comprising a computer program / instructions, characterized in that When the computer program / instructions are executed by a processor, the steps of a method for improving the safety of a multi-modal large model with modal decoupling as described in any one of claims 1 to 7 are implemented.

Images