Image reconstruction loss-based migration adversarial attack method and apparatus, and electronic device

By generating adversarial samples based on image reconstruction loss based on CLIP and MAE, the migration attack is carried out, which solves the limitations of the robustness evaluation of visual adversariality of visual language big models in the prior art, and realizes a universal attack on visual mode. Vulnerabilities in visual language models and improve robustness.

CN119027690AInactive Publication Date: 2024-11-26BEIJING ACAD OF ARTIFICIAL INTELLLIGENCE

Patent Information

Application Number
CN202411514235.2
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-10-29
Publication Date
2024-11-26
Estimated Expiration
Not applicable · inactive patent

AI Technical Summary

Technical Problem

The existing methods of evaluating the robustness of visual adversarial models for visual languages ​​are limited to specific tasks or situations, and it is difficult to effectively attack the model's understanding of the picture itself, and lacks universality and generalization.

Method used

Adversarial samples are generated by using decoder based on CLIP picture encoder and MAE architecture, and the adversarial samples are generated by maximizing image reconstruction losses and migrating them to visual language big models to perform white box attacks to detect their robustness.

Benefits of technology

This method can expand attacks on visual modals under different text prompts and scenarios, with good scalability, and evaluates the robustness of visual language models by evaluating the power of visual language models against samples, and discovers the vulnerabilities of its visual modality, providing Improve and fine-tune the direction.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119027690A_ABST
    Figure CN119027690A_ABST
Patent Text Reader

Abstract

The invention discloses a migration adversarial attack method and device based on picture reconstruction loss and electronic equipment, and belongs to the technical field of visual language large models. The method comprises the following steps: taking a CLIP picture encoder as an encoder and training a decoder based on an MAE architecture to obtain a trained encoder-decoder model; inputting an original picture into the encoder-decoder model, maximizing reconstruction loss of an output picture of the encoder-decoder model relative to the original picture based on a projection gradient descent method, and generating an adversarial sample; and migrating the adversarial sample to the visual language large model for attacking, and detecting the robustness of the visual language large model. According to the method, optimization is carried out by increasing the picture reconstruction loss, the adversarial sample is generated and used for attacking the visual language large model, the method can be expanded to different text prompts and scenes, and the method has good expansibility.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of large visual language models, and in particular to a migration counterattack method, device and electronic device based on image reconstruction loss. Background Art

[0002] A visual language big model refers to a big model that can process visual and language modal information; compared to a language big model, a visual language big model can combine information from the visual modality, so it can handle a wider range of problems in the real world. At present, visual language big models, such as LLaVA, Qwen-VL, GPT-4V, etc., have demonstrated excellent performance in a variety of tasks, such as image classification, visual question answering, and image-text matching. However, as more and more visual language big models are put into practical applications, their security and robustness have also attracted widespread attention and concerns. Users hope that the visual language big model can be robust enough, that is, the model can keep the results stable when facing small perturbations or noise. The robustness evaluation of the visual language big model is helpful for researchers and users to understand the reliability of the big model, which can avoid improper use and harmful results caused by the vulnerabilities of the big model to a certain extent, and provide direction and ideas for the further development and improvement of the visual language big model.

[0003] Adversarial attack refers to adding some tiny noise to the input of the model that generally does not have a significant impact on human judgment, in order to induce the model to make wrong decisions. Adversarial attack is one of the methods for evaluating the robustness of the model. By studying the degree to which the model is affected by adversarial samples, the robustness and vulnerabilities of the model are explored. Current adversarial attack methods can be divided into black-box adversarial attack methods and white-box adversarial attack methods. Black-box adversarial attack refers to the situation where the attack model is unknown, and the attack effect and targeting are often poor. The white-box attack method is the situation where the model structure and parameters are known, which is often difficult to achieve in real scenarios.

[0004] For large visual language models, both visual and textual modalities can be attacked. However, considering the high-dimensional and continuous nature of the visual modality, it is often a weak link in the attack. There are already some methods for evaluating the robustness of visual adversarial large models of visual language. For example, Cui et al. set a loss function based on the cosine similarity between the image encoder and the label text encoding when generating adversarial samples; Zhao et al. designed attacks based on the similarity between the image and the target text; these methods often combine information from the label or text modality when generating adversarial samples, which means that adversarial attacks may be limited to certain specific tasks or situations. Therefore, it is urgent to design an adversarial attack method that destroys the model's understanding of the image itself in order to improve the versatility and generalization of the method. Summary of the invention

[0005] In order to solve the problems existing in the prior art, the present invention provides the following technical solutions.

[0006] A first aspect of the present invention provides a migration countermeasure attack method based on image reconstruction loss, comprising: S101, using the CLIP image encoder as the encoder and training a decoder based on the MAE architecture to obtain a trained encoder-decoder model; S102, inputting the original image in the visual language task into the trained encoder-decoder model, and maximizing the reconstruction loss of the trained encoder-decoder model to generate adversarial samples; S103, transfer the adversarial sample to the visual language large model, attack the visual language large model, and detect the robustness of the visual language large model.

[0007] Preferably, obtaining the trained encoder-decoder model comprises: freezing the parameters of the picture encoder based on the CLIP model, and training the decoder based on the MAE architecture using the training data set to obtain the trained encoder-decoder model.

[0008] Preferably, the parameters of the picture encoder based on the CLIP model are frozen, and a decoder based on the MAE architecture is trained using a training data set to obtain a trained encoder-decoder model, including: occluding the original picture in the training data set according to a certain ratio, inputting the occluded original picture into the picture encoder of the CLIP model with frozen parameters to extract coding features, introducing mask marks into the coding features, that is, combining the coding features with the mask marks in the order in the original picture to construct a complete tag sequence; processing by a decoder with the same structure as MAE, reconstructing the picture in pixels, and training the decoder according to the picture reconstruction loss.

[0009] Preferably, in step S102, the reconstruction loss of the trained encoder-decoder model output image relative to the original image is maximized based on the projected gradient descent method.

[0010] Preferably, the reconstruction loss is the mean square error in pixel space between the original image and the corresponding reconstructed image output by the encoder-decoder model.

[0011] Preferably, maximizing the reconstruction loss of the output image of the trained encoder-decoder model relative to the original image based on the projected gradient descent method includes: adding random noise not exceeding a preset maximum disturbance amplitude to the original image to initialize the original image; gradually adding noise to the initialized original image, and in each iterative step, solving the gradient of the loss function relative to the image obtained in the previous step, and adding noise to the image obtained in the previous step according to the direction of gradient descent; in the process of adding noise at each step, the pixel-by-pixel difference between the image with added noise and the original image is limited to not exceed the value of the preset maximum disturbance amplitude.

[0012] A second aspect of the present invention provides a migration counterattack device based on image reconstruction loss, comprising: The model training module is used to use the CLIP image encoder as the encoder and train the decoder based on the MAE architecture to obtain a trained encoder-decoder model; The adversarial sample generation module is used to input the original images in the visual language task into the trained encoder-decoder model, and maximize the reconstruction loss of the output image of the trained encoder-decoder model relative to the original image to generate adversarial samples; The transfer attack module is used to transfer adversarial samples to the visual language large model, attack the visual language large model, and detect the robustness of the visual language large model.

[0013] A third aspect of the present invention provides a memory storing a plurality of instructions, wherein the instructions are used to implement the migration counterattack method based on image reconstruction loss as described in the first aspect.

[0014] A fourth aspect of the present invention provides an electronic device, comprising a processor and a memory connected to the processor, wherein the memory stores a plurality of instructions, which can be loaded and executed by the processor so that the processor can execute the migration counterattack method based on image reconstruction loss as described in the first aspect.

[0015] The beneficial effects of the present invention are as follows: the present invention provides a transfer adversarial attack method, device and electronic device based on image reconstruction loss. The transfer adversarial attack method includes using an open source white box model as a target model to generate adversarial samples. Specifically, the present invention uses the image encoder of the contrastive language-image pre-training (CLIP) model to generate visual adversarial samples; and then migrates the generated samples to the visual language large model for attack. In the process of generating adversarial samples, in order to better widen the gap between the original image and the adversarial image, the present invention is based on the idea of ​​reconstructing after occluding image pixels in the masked autoencoder (MAE), and optimizes by increasing the image reconstruction loss to generate adversarial samples for attack. The transfer adversarial attack method provided by the present invention performs adversarial attacks on visual modalities, which can be extended to different text prompts and scenarios, and has good scalability. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 This is a flow chart of the migration counterattack method based on image reconstruction loss according to the present invention; Figure 2 A schematic diagram of the training process of the decoder based on the MAE architecture of the present invention; Figure 3 A comparison diagram of a picture reconstructed by the trained encoder-decoder model of the present invention and the corresponding original picture; Figure 4 A schematic diagram of the process of generating adversarial samples using a trained encoder-decoder model according to the present invention; Figure 5 An example diagram of using adversarial samples generated by the present invention to attack a large visual language model; Figure 6 Schematic diagram of the functional structure of the migration anti-attack device based on image reconstruction loss described in the present invention. DETAILED DESCRIPTION

[0017] In order to better understand the above technical solution, the above technical solution will be described in detail below in conjunction with the accompanying drawings and specific implementation methods.

[0018] The method provided by the present invention can be implemented in the following terminal environment, and the terminal may include one or more of the following components: a processor, a memory, and a display screen. The memory stores at least one instruction, and the instruction is loaded and executed by the processor to implement the method described in the following embodiment.

[0019] The processor may include one or more processing cores. The processor uses various interfaces and lines to connect various parts of the entire terminal, and executes various functions of the terminal and processes data by running or executing instructions, programs, code sets or instruction sets stored in the memory, and calling data stored in the memory.

[0020] The memory may include random access memory (RAM) or read-only memory (ROM). The memory may be used to store instructions, programs, codes, code sets or instructions.

[0021] The display screen is used to display the user interface of each application.

[0022] In addition, those skilled in the art can understand that the structure of the above terminal does not constitute a limitation on the terminal, and the terminal may include more or fewer components, or combine certain components, or arrange the components differently. For example, the terminal also includes components such as a radio frequency circuit, an input unit, a sensor, an audio circuit, and a power supply, which will not be described in detail here.

[0023] Embodiment 1 like Figure 1 As shown, the embodiment of the present invention provides a migration counterattack method based on image reconstruction loss, including: S101, using the CLIP image encoder as the encoder and training a decoder based on the MAE architecture to obtain a trained encoder-decoder model; It should be noted that the trained encoder-decoder model includes a CLIP image encoder and a decoder based on the MAE architecture, wherein obtaining the trained encoder-decoder model includes freezing the parameters of the image encoder based on the CLIP model, and using the training data set to train the decoder based on the MAE architecture to obtain the trained encoder-decoder model.

[0024] In a preferred embodiment, the ImageNet dataset is used as the training dataset, wherein the ImageNet dataset is a computer vision dataset created by Professor Fei-Fei Li of Stanford University. Figure 2 As shown, Figure 2The snowflake in the middle indicates freezing the parameters of the corresponding module, and the flame indicates optimizing the parameters of the corresponding module during training. The specific training process is as follows: First, the original image in the ImageNet dataset is masked at a certain ratio (for example, 75%). Then, the masked original image is input into the image encoder of the CLIP model with frozen parameters for encoding feature extraction. Subsequently, the mask token is introduced into the encoding feature, that is, the encoding feature and the mask token are combined in the order in the original image to construct a complete tag sequence. Finally, the decoder with the same structure as MAE is processed to reconstruct the image in pixels. In the decoder training based on the MAE architecture, the AdamW optimizer is used, the basic learning rate is set to 0.01, the number of iterations is set to 50 times, of which the first 5 times are preheating, and the weight decay is set to 0.05. The decoder trained in this way can reconstruct the image based on the encoding features of the image encoder of the CLIP model without adversarial noise.

[0025] like Figure 3 As shown in Figure 1, the trained encoder-decoder model can achieve better reconstruction effect on the original image.

[0026] S102, inputting the original image into the trained encoder-decoder model, and maximizing the reconstruction loss of the output image of the encoder-decoder model relative to the original image to generate an adversarial sample; In a preferred embodiment, the reconstruction loss of the output image of the trained encoder-decoder model relative to the original image is maximized based on the projected gradient descent method. The process of generating adversarial samples using the trained encoder-decoder model is as follows: Figure 4 As shown, Figure 4 The snowflakes in the middle represent the freezing of the parameters of the corresponding modules. It should be noted that the original image in the visual language task is the object to be disturbed. Specifically, based on the trained encoder-decoder model, noise within a certain range is added to the original image in the visual language task to widen the gap between the original image and the adversarial image for the target model, destroying the target model's perception of the image information, so as to achieve the effect of attacking the image encoder of the CLIP model. The above purpose is achieved by optimizing the reconstruction loss through the projected gradient descent method. The initial reconstruction loss is obtained through the reconstruction process of the encoder-decoder model structure obtained by training in step S101.

[0027] In a preferred embodiment, the original image The reconstruction loss is the mean square error (MSE) in pixel space between the original image and the reconstructed image output by the trained encoder-decoder structure, as shown in formula (1): ; in, Represents the original image, Represents a reconstructed image. The sizes of the original image and the reconstructed image are ; Here only the loss in the mask image block is calculated, are the pixel coordinates in the mask image block.

[0028] When facing the original image in the visual language task, the encoder-decoder trained in step S101, that is, the trained encoder-decoder model, can reconstruct the image well, that is, it has a small reconstruction loss. In order to generate adversarial samples, the projected gradient descent (PGD) method is then used to optimize the adversarial image to maximize the reconstruction loss function. After gradually adding adversarial noise using the PGD method, as the reconstruction loss increases, it also means that the target model of the attack, that is, the image encoder of the CLIP model, perceives the adversarial image differently from the original image. The specific optimization process based on the PGD method is as follows: first, add random noise that does not exceed the preset maximum perturbation amplitude to the original image, initialize the original image, and then gradually add noise to the initialized original image. In each iterative step, solve the gradient of the loss function relative to the image obtained in the previous step, and add noise to the image obtained in the previous step according to the direction of gradient descent; in each step of adding noise, the pixel-by-pixel difference between the image with added noise and the original image is limited to not exceed the value of the preset maximum perturbation amplitude. The process is shown in formula (2). ; in Represents the original image, represents random noise, represents the initially generated noisy samples, Indicates The adversarial examples generated by the step, Indicates The adversarial examples generated by the step, represents the amount of disturbance added in each iteration, represents the symbolic function, Represents adversarial examples The reconstruction loss of Express Find the gradient, Represents the clipping process. Clipping is used to prevent the image from exceeding the valid range of [0,1] during the optimization process and satisfy and The difference between corresponding pixels does not exceed the preset maximum disturbance amplitude In a preferred embodiment, the maximum perturbation amplitude of the adversarial sample is set to , the total number of iteration steps , the perturbation amount in each iteration .

[0029] S103, transfer the adversarial samples to the large visual language model for attack and detect the robustness of the large visual language model.

[0030] It should be noted that in this step, the adversarial sample generated by the picture encoder of the CLIP model as the target model is migrated to the visual language large model to attack the visual language large model. Specifically, the generated adversarial picture plus the original text prompt is used as the input of the visual language large model, and the changes in the response of the visual language large model are observed. It is found that the accuracy of the response of the visual language large model is significantly lower than that of the original picture used in the visual language task. This shows that the visual language large model still has visual robustness loopholes, and also illustrates the effectiveness of the attack method of the present invention. In addition, the present invention mainly conducts adversarial attacks on visual modalities, which can be extended to different text prompts and scenes, and has good scalability. For example, for the same adversarial picture, using different text prompts and asking different questions can still achieve attacks.

[0031] The visual language large model can be LLaVA, Qwen-VL, GPT-4V or other visual language large models. In a preferred embodiment, the visual language large model selects the LLaVA model, and uses the images in the two visual language datasets MMBench and TextVQA as the original images used in the visual language task; MMBench is a multiple-choice question, which includes multiple evaluation dimensions such as target detection, text recognition, action recognition, image understanding, and relational reasoning; TextVQA is a short-answer question, and the picture contains text information. In addition, the attack method provided by the present invention can also be applied to other large visual language models and datasets. Figure 5 An example diagram of an adversarial sample generated by this method attacking a large visual language model is shown in FIG. 1 , and Table 1 quantitatively shows the impact of the attack method provided by the preferred embodiment on the large visual language model.

[0032] Table 1 Attack effects on large visual language models on different datasets Dataset Original sample accuracy (%) Adversarial sample accuracy (%) Accuracy decrease (%) MMBench 67.44 47.59 19.85 TextVQA 50.17 24.23 25.94 It can be seen from Table 1 that the attack method provided by the preferred embodiment shows obvious attack effects in different scenarios and tasks; in other words, the performance of the large visual language model is significantly reduced after the attack, which shows the effectiveness of the above-mentioned attack method, and at the same time shows that the existing large visual language model still has loopholes in visual adversarial robustness.

[0033] Through the adversarial attack method, the robustness of the visual language model can be evaluated, and the loopholes in the visual language model can be found, which is helpful to explore the future improvement and research direction of the visual language model. At the same time, the generated adversarial samples can also be used for adversarial training and fine-tuning of the visual language model, thereby improving the adversarial robustness of the visual language model. Compared with other adversarial attack methods, the present invention has the following advantages: First, the attack is mainly designed for visual image modality, and is based on image feature reconstruction, focusing on the model's destruction of image perception, without using label or text modality information, which means that our method has the potential to be extended to different text prompts, scenes and tasks, so the robustness of the visual language model in different scenes can be tested. Second, the present invention adopts a migration adversarial attack method. Compared with the pure white box adversarial attack method, which requires obtaining all the structural and parameter information of the visual language model (which is often very difficult), the method provided by the present invention is easier to implement and can be easily migrated to different visual language models, and has stronger practicality.

[0034] Embodiment 2 like Figure 6 As shown, another aspect of the present invention also includes a functional module architecture that is completely consistent with the aforementioned method flow, that is, an embodiment of the present invention further provides a migration anti-attack device based on image reconstruction loss, including: A model training module 201 is used to use the CLIP picture encoder as an encoder and train a decoder based on the MAE architecture to obtain a trained encoder-decoder model; An adversarial sample generation module 202 is used to input the original image into the trained encoder-decoder model and maximize the reconstruction loss of the encoder-decoder model output image relative to the original image to generate an adversarial sample; The migration attack module 203 is used to migrate the adversarial samples to the visual language large model for attacking and detecting the robustness of the visual language large model.

[0035] The device can be implemented by the migration counterattack method based on image reconstruction loss provided in the above-mentioned embodiment 1. The specific implementation method can be found in the description of embodiment 1, which will not be repeated here.

[0036] The present invention also provides a memory storing a plurality of instructions, wherein the instructions are used to implement the migration counterattack method based on image reconstruction loss as described in the first embodiment.

[0037] The present invention also provides an electronic device, comprising a processor and a memory connected to the processor, wherein the memory stores a plurality of instructions, and the instructions can be loaded and executed by the processor so that the processor can execute the migration counterattack method based on image reconstruction loss as described in Example 1.

[0038] Although preferred embodiments of the present invention have been described, additional changes and modifications may be made to these embodiments by those skilled in the art once the basic inventive concepts are known. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present invention. Obviously, those skilled in the art may make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

Claims

1. A transfer adversarial attack method based on image reconstruction loss, It is characterized in that include: S101, using the CLIP image encoder as the encoder and training a decoder based on the MAE architecture to obtain a trained encoder-decoder model; S102, inputting the original image into the encoder-decoder model, and maximizing the reconstruction loss of the output image of the encoder-decoder model relative to the original image, to generate an adversarial sample; S103, transferring the adversarial sample to the large visual language model for attack, and detecting the robustness of the large visual language model.

2. The transfer counterattack method based on image reconstruction loss as claimed in claim 1, It is characterized in that The obtaining of the trained encoder-decoder model comprises: freezing the parameters of the picture encoder based on the CLIP model, and training the decoder based on the MAE architecture using the training data set to obtain the trained encoder-decoder model.

3. The transfer counterattack method based on image reconstruction loss as claimed in claim 2, It is characterized in that The parameters of the picture encoder based on the CLIP model are frozen, and a decoder based on the MAE architecture is trained using a training data set to obtain a trained encoder-decoder model, including: Occlude the original image in the training dataset according to a certain ratio; Extracting encoding features from the picture encoder of the CLIP model whose input parameters of the occluded original picture are frozen; Introducing mask marks into the coding features, that is, combining the coding features with the mask marks in the order in the original image to construct a complete mark sequence; The decoder with the same structure as MAE is used for processing, the image is reconstructed in pixels, and the decoder is trained based on the image reconstruction loss.

4. The transfer counterattack method based on image reconstruction loss as claimed in claim 1, It is characterized in that In step S102, the reconstruction loss of the encoder-decoder model output picture relative to the original picture is maximized based on the projected gradient descent method.

5. The transfer counterattack method based on image reconstruction loss as claimed in claim 1, It is characterized in that The reconstruction loss is the mean square error in pixel space between the original image and the corresponding reconstructed image output by the encoder-decoder model.

6. The transfer counterattack method based on image reconstruction loss as claimed in claim 4, It is characterized in that The method of maximizing the reconstruction loss of the encoder-decoder model output picture relative to the original picture based on the projected gradient descent method includes: Add random noise that does not exceed the preset maximum disturbance amplitude to the original image to initialize the original image; Gradually add noise to the initialized original image. In each iteration step, solve the gradient of the loss function relative to the image obtained in the previous step, and add noise to the image obtained in the previous step according to the direction of gradient descent. In each step of adding noise, the pixel-by-pixel difference between the image with added noise and the original image is limited to not exceed the value of the preset maximum disturbance amplitude.

7. A transfer adversarial attack device based on image reconstruction loss, It is characterized in that include: The model training module is used to use the CLIP image encoder as the encoder and train the decoder based on the MAE architecture to obtain a trained encoder-decoder model; An adversarial sample generation module, used to input the original image into the encoder-decoder model and maximize the reconstruction loss of the output image of the encoder-decoder model relative to the original image to generate an adversarial sample; The migration attack module is used to migrate the adversarial sample to the large visual language model for attacking and detecting the robustness of the large visual language model.

8. A memory, It is characterized in that Multiple instructions are stored, and the instructions are used to implement the migration counterattack method based on image reconstruction loss as described in any one of claims 1-6.

9. An electronic device, It is characterized in that It includes a processor and a memory connected to the processor, the memory stores a plurality of instructions, and the instructions can be loaded and executed by the processor, so that the processor can execute the migration adversarial attack method based on image reconstruction loss as described in any one of claims 1-6.

Citation Information

Patent Citations

  • Data privacy protection method based on adversarial learning

    CN115936958A

  • Vision-language pre-training model-oriented multi-modal migration countermeasure attack method

    CN117094000A

  • Model pre-training method and device, image processing method and device, equipment and storage medium

    CN117523334A

  • AI model security assessment method based on collaborative multi-modal interaction

    CN118468278A

  • Masked Autoencoders for Computer Vision

    US20240096072A1

Cited By

  • Black box attack confrontation sample generation method and device

    CN119398131A