An optimization-based multi-modal LLM adversarial attack method in a physical environment

Abstract

The application belongs to the technical field of security of large language models, and discloses a kind of physical environment based on optimization multimodal LLM counterattack method, comprising: selecting image dataset, image feature extraction and visualization work are carried out, and the patch after random rotation is pasted to clean image to generate countermeasure sample, and normal distribution enhancement and momentum enhancement gradient are added to countermeasure sample area;The similarity of the calculated countermeasure sample and the target picture is calculated and the patch is updated iteratively, and the patch after circulation is pasted on clean image to generate countermeasure sample, and the CLIP score between the generated countermeasure sample and the predefined target text is calculated.The application selects a region in the image that corresponds to the patch size for disturbance, thereby achieving a significant attack effect in a physical environment, which gives the large language model stronger robustness, making the adversarial research more valuable for in-depth study.

Description

Technical Field

[0001] This invention belongs to the field of security technology of large language models, specifically relating to an optimized multimodal LLM adversarial attack method in a physical environment. Background Technology

[0002] With the widespread application of multimodal language and vision models (LLMs) in physical environments such as autonomous driving, intelligent monitoring, and robot navigation, their role in perception, understanding, and decision-making tasks is becoming increasingly important.

[0003] However, research has shown that adding adversarial patches to the input image, which are imperceptible to humans, can cause LLM to fail to produce accurate output results.

[0004] Some of these methods have also demonstrated that these attacks are feasible in the real world, namely by modifying an object and photographing it with a camera, then generating an adversarial patch on the object's known structure, such perturbation images, when input into an LLM, will result in a significant change in the semantic content.

[0005] Research has found that existing adversarial attack methods such as FGSM, PGD, and adding random global perturbations share a common problem: their attack effects are relatively weak in physical environments, and only become relatively significant when global perturbation attacks are performed. Summary of the Invention

[0006] To address the aforementioned technical problems, this invention provides an optimized multimodal LLM adversarial attack method in a physical environment. Unlike global perturbation, this invention selects a region in the image that matches the patch size for perturbation, thereby achieving a significant attack effect in a physical environment. This characteristic gives large language models stronger robustness, making adversarial research more valuable.

[0007] To achieve the above objectives, the present invention is implemented through the following technical solution:

[0008] This invention is an optimized multimodal LLM adversarial attack method in a physical environment, which includes the following steps:

[0009] Step 1: Select an image encoder f φ (x) and a text encoder As a proxy model, namely the CLIP proxy model, and will c tar This is represented as the target response returned by the h victim model;

[0010] Step 2: Preprocess the image by converting the input image into the format required by the CLIP proxy model, and apply the patch to the image using a random conversion method;

[0011] Step 3: Based on the image processed in Step 2, use the normal distribution enhancement update strategy and the baseline method to randomly search for appropriate hostile perturbations using the common distribution.

[0012] Step 4: Based on the image processed in Step 2, replace the normal distribution enhancement update strategy in Step 3 with a momentum-based adversarial perturbation update strategy.

[0013] Step 5: Encode the image processed in Step 2 to obtain the feature vector x of the processed image. adv , will the proxy model's c tar and x adv The features are matched, x adv Should meet

[0014]

[0015] ||x cle -x adv || p ≤ε

[0016] In the formula, the image encoder and text encoder are chosen to have the same output size, and their internal product indicates that c tar and x adv The cross-channel similarity constraint optimization problem is solved using the projective gradient descent (PGD) method.

[0017] Step 6: Input the adversarial example into the LLM, perform image-to-text conversion, and store the generated response;

[0018] Step 7: Use different types of CLIP proxy model text encoders to evaluate the response generated in Step 6 and the predefined target response c. tar The similarity between them.

[0019] A further improvement of the present invention is that the image preprocessing method in step 2 is specifically as follows:

[0020] Step 2.1.1: Resize, center-crop, convert to RGB format, and convert to tensor form of the input image: Use the Resize image transformation class in PyTorch's torchvision library to resize the input image to the same resolution as the visual input of the CLIP model. Use the bicubic interpolation method (BICUBIC) for interpolation. Use the CenterCrop image cropping utility class in PyTorch's torchvision library to crop the image from the center to the same size as the visual input resolution of the CLIP proxy model. Open the image using the PIL library in Python, and then convert the image to RGB format to a tensor using the convert_to_tensor method of NumPy arrays.

[0021] Step 2.1.2: Normalize the image: Adjust the image processed in step 2.1.1 by normalizing the data within the range to reduce image region selection errors caused by specific data.

[0022] A further improvement of the present invention is that step 2, which uses a random conversion method to apply the patch to the image, specifically includes the following steps:

[0023] Step 2.2.1: The original image and the patch image to be added are loaded;

[0024] Step 2.2.2: A random transformation method of rotation, scaling or flipping is selected, and corresponding transformation parameters are randomly generated. Random noise is placed on top of the patch to randomly change the brightness and contrast of the image. These random parameters are applied to the patch image to make it undergo transformation.

[0025] Step 2.2.3: Randomly select a location on the original image to ensure that the patch does not extend beyond the image boundary;

[0026] Step 2.2.4: The converted patched image is overlaid on a selected position of the original image to generate a new image containing the patched information.

[0027] A further improvement of the present invention is that the normal distribution enhancement update strategy used in step 3 is specifically as follows:

[0028] I adv =I adv_prev +P final

[0029]

[0030] Among them, I adv and I adv_prev The attack image refers to the difference between the current attack step and previous attack steps, P.final This indicates the final counter-perturbation at the current attack step, P org_normal It is the original adversarial perturbation baseline, P add_normal It is an additional counter-perturbation.

[0031] A further improvement of the present invention is that, in step 4, the momentum-based adversarial perturbation update strategy specifically includes the following steps:

[0032] Step 4.1: Adjust the pixel value range of the image processed in Step 2 to between 0 and 1;

[0033] Step 4.2: Normalize the data within the pixel value range to reduce image region selection errors caused by specific data;

[0034] Step 4.3: Based on adversarial examples, replace the normal distribution enhancement update strategy with a momentum-based adversarial perturbation update strategy:

[0035]

[0036] P add_m =θ*P add_m_cur +(1-θ)*P add_m_mean ,θ∈(0,1)

[0037] P add_m =θ*P add_m_cur +(1-θ)*P add_m_mean ,θ∈(0,1)

[0038] In the formula P add_m The final momentum-enhancing adversarial perturbation is added to the original random normal 1. The momentum-enhancing adversarial perturbation θ controls the effect of the current adversarial perturbation direction and the average direction of all previous adversarial perturbation calculations. θ is set in the range of 0 to 1.

[0039] Step 4.4: Calculate the gradient against perturbation using the backpropagation (backward()) algorithm:

[0040]

[0041] In the formula, It is calculated layer by layer using the chain rule;

[0042] Step 4.5: Use non-printable scoring loss L nps The loss is calculated to ensure that the normal distribution enhancement update strategy favors images with smooth color transitions and prevents noisy images, as expressed as:

[0043]

[0044] In the formula, Ppatch It is a pixel in patch P, c print It is one of the printable colors c;

[0045] Step 4.6: Calculate the total change L of the image based on patch P. tv :

[0046]

[0047] If adjacent pixels are similar, the score is low; if adjacent pixels are different, the score is high.

[0048] Step 4.7, the total loss function is as follows:

[0049] L=αL nps +βL tv

[0050] In the formula, α and β are both hyperparameters, and L is scaled using hyperparameters α and β respectively. tv and L nps Then, L is optimized using a momentum-based adversarial perturbation update strategy, thus freezing all weights and biases in the CLIP proxy model and updating only the pixel values ​​of the adversarial patch, with the initial patch p initialized with random values ​​at the start of the optimization process.

[0051] The beneficial effects of this invention are:

[0052] This invention uses a proxy model to randomly apply patches to an image, making the perturbation more targeted.

[0053] The invention demonstrates, through testing the printed patch in LLM, that the patch works well with hidden text detectors, suggesting that security systems using similar detectors may be vulnerable to this type of attack.

[0054] The method proposed in this invention, which uses local patching for perturbation, has an attack effect comparable to that of a global perturbation attack.

[0055] Unlike global perturbation, this invention selects a region in the image that matches the size of the patch for perturbation, thereby achieving a significant attack effect in the physical environment. This characteristic gives large language models stronger robustness, making adversarial research more valuable. Attached Figure Description

[0056] Figure 1 This is a flowchart illustrating the principle of the present invention. Detailed Implementation

[0057] The embodiments of the present invention will be disclosed below with reference to the drawings. For clarity, many practical details will be described in the following description. However, it should be understood that these practical details are not intended to limit the invention. That is, in some embodiments of the invention, these practical details are not essential.

[0058] This invention aims to generate adversarial perturbation patches targeting a class of inherently variable targets, namely the human body. The strategy seeks to induce a fundamental shift in the semantics of the LLM's output by inputting an image carrying the perturbation patch into the LLM, thereby successfully concealing the target from detection by the LLM text detector. This attack could be abused by malicious actors, for example, to deceive surveillance systems by holding a small piece of cardboard in front of the body to obscure the surveillance camera, thus avoiding detection and circumventing alarm systems through the text content output by the LLM.

[0059] like Figure 1 As shown, this invention is an optimized multimodal LLM adversarial attack method in a physical environment, characterized by the following steps:

[0060] Step 1: Since the victim model is visual and linguistic, select an image encoder f. φ (x) and a text encoder As a proxy model, and will c tar The target response returned by the victim model is represented by h. The proxy model selected in this invention is the CLIP proxy model.

[0061] Step 2: Preprocess the image by converting the input image into the format required by the CLIP proxy model, and apply the patch to the image using a random conversion method.

[0062] The specific image preprocessing methods are as follows:

[0063] Step 2.1.1: Resize and center-crop the input image, then convert it to RGB format and tensor form. Specifically: Use the Resize image transformation class in PyTorch's torchvision library to resize the input image to the same resolution as the CLIP model's visual input; use bicubic interpolation (BICUBIC) for interpolation; use the CenterCrop image cropping utility class in PyTorch's torchvision library to center-crop the image to the same size as the CLIP proxy model's visual input resolution; open the image using Python's PIL library; and then convert the RGB format image to a tensor using the NumPy array's convert_to_tensor method.

[0064] Step 2.1.2: Image Normalization: Adjust the image processed in Step 2.1.1 by normalizing the data within the specified range to reduce image region selection errors caused by specific data. The purpose of this step is to subtract the mean from the pixel values ​​of the image and divide by the standard deviation to ensure that each channel of the image has a near-zero mean and similar scale. The mean and standard deviation values ​​used here are pre-calculated values ​​for the CLIP model used for image standardization.

[0065] Step 2, which uses a random transformation method to apply the patch to the image, involves the following steps: First, the original image and the image to be patched are loaded. Then, a random transformation method is selected, such as rotation, scaling, or flipping, and corresponding transformation parameters are randomly generated. In this invention, the patch is rotated 20 degrees in each direction, randomly scaled, and random noise is placed on top of the patch to randomly vary the image's brightness and contrast. These random parameters are applied to the patch image, causing it to undergo transformation. Next, a random position is selected on the original image to ensure that the patch does not extend beyond the image boundaries. Finally, the transformed patch image is superimposed on the selected position in the original image, generating a new image that includes the information about the added patch. Repeated application of this method, combined with different transformation methods and parameters, can produce diverse image samples, helping to improve the model's robustness and generalization performance.

[0066] Step 3: Based on the image processed in Step 2, a normal distribution enhancement and update strategy is used, employing a baseline method and randomly searching for appropriate hostile perturbations using a common distribution. Specifically, the normal distribution enhancement and update strategy is as follows:

[0067] I adv =I adv_prev +P final

[0068]

[0069] Among them, I adv and I adv_prev The attack image refers to the difference between the current attack step and previous attack steps, P. final This indicates the final counter-perturbation at the current attack step, P org_normal It is the original adversarial perturbation baseline, P add_normal It is an additional counter-perturbation.

[0070] Step 4: Based on the image processed in Step 2, replace the normal distribution enhancement update strategy in Step 3 with a momentum-based adversarial perturbation update strategy. The momentum-based adversarial perturbation update strategy specifically includes the following steps:

[0071] Step 4.1: Adjust the pixel value range of the image processed in Step 2 to between 0 and 1;

[0072] Step 4.2: Normalize the data within the pixel value range to reduce image region selection errors caused by specific data;

[0073] Step 4.3: Based on adversarial examples, replace the normal distribution enhancement update strategy with a momentum-based adversarial perturbation update strategy:

[0074]

[0075] P add_m =θ*P add_m_cur +(1-θ)*P add_m_mean , θ∈(0,1)

[0076] P add_m =θ*P add_m_cur +(1-θ)*P add_m_mean , θ∈(0,1)

[0077] In the formula P add_m The final momentum-enhancing adversarial perturbation is added to the original random normal 1. The momentum-enhancing adversarial perturbation θ controls the effect of the current adversarial perturbation direction and the average direction of all previous adversarial perturbation calculations. θ is set in the range of 0 to 1.

[0078] The momentum-based adversarial perturbation update process includes momentum initialization, loss function calculation, gradient calculation, and parameter update.

[0079] Step 4.4: Calculate the gradient against perturbation using the backpropagation (backward()) algorithm:

[0080]

[0081] In the formula, It is calculated layer by layer using the chain rule;

[0082] Step 4.5: Use non-printable scoring loss L nps The loss is calculated to ensure that the normal distribution enhancement update strategy favors images with smooth color transitions and prevents noisy images. This loss is a factor representing whether the colors in the patch can be well represented by a regular printer. It is expressed as:

[0083]

[0084] In the formula, P patch It is a pixel in patch P, c printIt is one of the printable colors c; this loss is closely related to the colors in the image and to the set of printable colors.

[0085] Step 4.6: Calculate the total change L of the image based on patch P. tv :

[0086]

[0087] If adjacent pixels are similar, the score is low; if adjacent pixels are different, the score is high.

[0088] Step 4.7, the total loss function is as follows:

[0089] L=αL nps +βL tv

[0090] In the formula, α and β are both hyperparameters, and L is scaled using hyperparameters α and β respectively. tv and L nps Then, L is optimized using a momentum-based adversarial perturbation update strategy, thus freezing all weights and biases in the CLIP proxy model and updating only the pixel values ​​of the adversarial patch, with the initial patch p initialized with random values ​​at the start of the optimization process.

[0091] Step 5: Encode the image processed in Step 2 to obtain the feature vector x of the processed image. adv , will the proxy model's c tar and x adv The features are matched, x adv Should meet

[0092]

[0093] ||x cle -x adv || p ≤ε

[0094] In the formula, the image encoder and text encoder are chosen to have the same output size, and their internal product indicates that c tar and x adv The cross-channel similarity constraint optimization problem is solved using the projective gradient descent (PGD) method.

[0095] Step 6: Input the adversarial example into the LLM, perform image-to-text conversion, and store the generated response;

[0096] Step 7: Use different types of CLIP proxy model text encoders to evaluate the response generated in Step 6 and the predefined target response c. tar The similarity between them.

[0097] To verify whether the present invention has significant advantages, the present invention selects 50K clean images x from the ImageNet-1K validation set. cle And randomly select the target response c from the MS-COCO header for each clean image. tar On the MS-COCO-100 dataset, this invention applies the proposed method to generate adversarial example images on clean images, using input images computed by various CLIP text encoders and their ensembles, and predefined target responses c. tar The CLIP scores between the two groups are shown in Table 1.

[0098] Table 1 Input Image and Predefined Target Response c tar Experimental results of CLIP score Rank-1 between

[0099]

[0100] The above description is only a preferred embodiment of the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A method for adversarial attacks based on optimization in a physical environment using multimodal LLM, characterized in that: The multimodal LLM adversarial attack method includes the following steps: Step 1: Select an image encoder and a text encoder As a proxy model, namely the CLIP proxy model, and will This is represented as the target response returned by the h victim model; Step 2: Preprocess the image by converting the input image into the format required by the CLIP proxy model, and apply the patch to the image using a random conversion method; Step 3: Based on the image with added patch information processed in Step 2, use the normal distribution enhancement update strategy and the baseline method to randomly search for appropriate hostile perturbations using the common distribution. Step 4: Based on the image processed in Step 2, replace the normal distribution enhancement update strategy in Step 3 with a momentum-based adversarial perturbation update strategy. Step 5: Encode the image processed in Step 4 to obtain the feature vector of the processed image. The proxy model and Matching the features, Should meet In the formula, the image encoder and text encoder are chosen to have the same output size, and their internal product indicates that... and The cross-channel similarity constraint optimization problem is solved using the projective gradient descent (PGD) method. Step 6: Input the adversarial example into the LLM, perform image-to-text conversion, and store the generated response; Step 7: Use a text encoder to evaluate the response generated in Step 6 and the predefined target response. The similarity between them, where: The normal distribution enhancement update strategy used in step 3 is as follows: in, and This refers to the attack image in the current attack step compared to previous attack steps. This indicates the final counter-perturbation in the current attack step. It is the original counter-perturbation baseline. It is an additional counter-perturbation; Step 4, the momentum-based adversarial perturbation update strategy specifically includes the following steps: Step 4.1: Adjust the pixel value range of the image processed in Step 2 to between 0 and 1; Step 4.2: Normalize the data within the pixel value range to reduce image region selection errors caused by specific data; Step 4.3: Based on adversarial examples, replace the normal distribution enhancement update strategy with a momentum-based adversarial perturbation update strategy: In the formula To add a final momentum boost to the original random normal 1 to counteract the perturbation, momentum boost counteracts the perturbation. The effect of controlling the current direction of the adversarial perturbation and the average direction of all previous adversarial perturbations. Set within the range of 0 to 1; Step 4.4: Calculate the gradient against perturbation using the backpropagation (backward()) algorithm: In the formula, It is calculated layer by layer using the chain rule; Step 4.5: Use non-printable scoring loss L nps The loss is calculated to ensure that the normal distribution enhancement update strategy favors images with smooth color transitions and prevents noisy images, as expressed as: In the formula, P patch It is a pixel in patch P, c print It is one of the printable colors c; Step 4.6: Calculate the total change L of the image based on patch P. tv : If adjacent pixels are similar, the score is low; if adjacent pixels are different, the score is high. Step 4.7, the total loss function is as follows: In the formula, α and β are both hyperparameters, and L is scaled using hyperparameters α and β respectively. tv and L nps Then, L is optimized using a momentum-based adversarial perturbation update strategy, thus freezing all weights and biases in the CLIP proxy model and updating only the pixel values ​​of the adversarial patch, with the initial patch p initialized with random values ​​at the start of the optimization process.

2. The method for analyzing multimodal LLM adversarial attacks in a physical environment according to claim 1, characterized in that: The image preprocessing method in step 2 is as follows: Step 2.1.1: Resize, center-crop, convert to RGB format, and convert to tensor form of the input image: Use the Resize image transformation class in PyTorch's torchvision library to resize the input image to the same resolution as the visual input of the CLIP model. Use the bicubic interpolation method BICUBIC for interpolation. Use the CenterCrop image cropping utility class in PyTorch's torchvision library to crop the image from the center to the same size as the visual input resolution of the CLIP proxy model. Open the image using the PIL library in Python, and then convert the image to RGB format to a tensor using the convert_to_tensor method of NumPy arrays. Step 2.1.2: Normalize the image: Adjust the image processed in step 2.1.1 by normalizing the data within the range to reduce image region selection errors caused by specific data.

3. The method for adversarial attacks based on optimization in a physical environment according to claim 2, characterized in that: Step 2, which uses a random transformation method to apply the patch to the image, specifically includes the following steps: Step 2.2.1: The original image and the patch image to be added are loaded; Step 2.2.2: A random transformation method of rotation, scaling or flipping is selected, and corresponding transformation parameters are randomly generated. Random noise is placed on top of the patch to randomly change the brightness and contrast of the image. These random parameters are applied to the patch image to make it undergo transformation. Step 2.2.3: Randomly select a location on the original image to ensure that the patch does not extend beyond the image boundary; Step 2.2.4: The converted patched image is overlaid on a selected position of the original image to generate a new image containing the patched information.

Images