Method and electronic device for vision-language models against jailbreak attacks

Abstract

This invention proposes a method and electronic device for resisting jailbreak attacks. The method includes: receiving input text and an input image; generating cleaned text based on the input text; inputting the cleaned text into a suffix generator to obtain a suffix, and appending the suffix to the cleaned text to obtain reinforced text; generating a cleaned image based on the input image; inputting the reinforced text and the cleaned image into a visual language model to obtain a response; inputting the response and a prompt into a judgment model to obtain a reward, which represents whether the response is a positive response or a jailbreak response; and adjusting the suffix generator based on the reward.

Claims

1. A method for resisting jailbreak attacks, comprising: receiving an input text and an input image; generating cleaned text based on the input text; inputting the cleaned text into a suffix generator to obtain a suffix, and appending the suffix to the cleaned text to obtain reinforced text; generating a cleaned image based on the input image; inputting the reinforced text and the cleaned image into a visual language model to obtain a response; inputting the response and a first prompt into a judgment model to obtain a reward, the reward representing whether the response is a benign response or a jailbreak response; and adjusting the suffix generator based on the reward.

2. The method as described in claim 1, wherein the step of generating the cleaned text based on the input text includes: The input text and a second prompt are fed into a language model to obtain the cleaned text.

3. The method as described in claim 1, wherein the step of generating the cleaned image based on the input image includes: Multiple forward iterations are performed to add noise to the input image to generate a noisy image; And perform multiple reverse iterations to remove noise from the noisy image to obtain the cleaned image.

4. The method as described in request item 1, wherein the step of adjusting the suffix generator based on the reward includes: Add the reward to an objective function; The KL divergence between the output distribution of the suffix generator and the output distribution of a reference model is added to the objective function.

5. The method as described in claim 4, wherein the objective function is expressed as the following mathematical expression, where is the cleansed text, xsuffix is ​​the suffix, y is the response of the visual language model, R(y) is the reward, β is a parameter, π() is the suffix generator, πref() is the reference model, and DKL is a function used to calculate the KL divergence.

6. The method as described in claim 1, wherein the cleaned text is the same as the input text, or the cleaned image is the same as the input image.

7. An electronic device comprising: a memory for storing a plurality of instructions; and a processor electrically connected to the memory for executing the instructions to complete a plurality of steps: receiving an input text and an input image; generating cleaned text based on the input text; inputting the cleaned text to a suffix generator to obtain a suffix, and appending the suffix to the cleaned text to obtain enhanced text; generating a cleaned image based on the input image; inputting the enhanced text and the cleaned image to a visual language model to obtain a response; inputting the response and a first prompt to a judgment model to obtain a reward, the reward representing whether the response is a positive response or a jailbreak response; and adjusting the suffix generator based on the reward.

8. The electronic device as claimed in claim 7, wherein the step of generating the cleaned text based on the input text includes: The input text and a second prompt are fed into a language model to obtain the cleaned text.

9. The electronic device as claimed in claim 7, wherein the step of generating the cleaned image based on the input image includes: Multiple forward iterations are performed to add noise to the input image to generate a noisy image; And perform multiple reverse iterations to remove noise from the noisy image to obtain the cleaned image.

10. The electronic device as claimed in claim 7, wherein the step of adjusting the suffix generator based on the reward includes: Add the reward to an objective function; The KL divergence between the output distribution of the suffix generator and the output distribution of a reference model is added to the objective function.

11. The electronic device as claimed in claim 10, wherein the objective function is expressed as the following mathematical expression, where is the cleansed text, xsuffix is ​​the suffix, y is the response of the visual language model, R(y) is the reward, β is a parameter, π() is the suffix generator, πref() is the reference model, and DKL is a function used to calculate the KL divergence.

12. The electronic device as claimed in claim 7, wherein the cleaned text is the same as the input text, or the cleaned image is the same as the input image.

Images