Sample purification method, device, equipment and medium

Abstract

The present application discloses a sample purification method, device, equipment and medium, which belongs to the field of artificial intelligence technology. The method includes: obtaining an adversarial sample, which is a sample generated after performing an adversarial attack on a clean sample, and the adversarial sample includes an adversarial image and text; adding a first noise to the adversarial image to obtain a noisy image; calculating the noisy cross-modal attention corresponding to the noisy image, and the noisy cross-modal attention is used to describe the attention distribution of the noisy image and the text; adding a second noise to the adversarial image based on the noisy cross-modal attention to obtain a purified image. The present application is guided by cross-modal attention, and purification is achieved by adding noise to the adversarial image. This purification method not only processes adversarial images, but also optimizes the cross-modal attention mechanism itself. The visual language model can obtain correct prediction results based on the purified image, and this purification method enhances the robustness of the visual language model.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of artificial intelligence technology, and in particular to a sample purification method, device, equipment and medium. Background Art

[0002] Large Vision-Language Models (LVLMs) are broadly defined as multimodal models that can learn from images and text. Large Vision-Language Models (VLMs) are vulnerable to adversarial examples when processing tasks. Adversarial examples are samples that make minor modifications or perturbations to clean samples, causing the VLM to make incorrect predictions.

[0003] In related technologies, adversarial samples are purified by performing some preprocessing operations on the image input into the visual language model. For example, the image is smoothed, that is, a smoothing filter is applied to reduce image noise, thereby reducing the impact of adversarial attacks; another example is the image is scaled, that is, the image features are changed by changing the scale of the image (such as zooming), thereby reducing the effectiveness of adversarial attacks.

[0004] However, the purification of adversarial samples in related technologies mainly relies on traditional image processing technology, resulting in unsatisfactory purification effects for adversarial samples. In practical applications, adversarial samples will still have an impact on the task output (prediction results) of the visual language model. Summary of the invention

[0005] The present application provides a sample purification method, device, equipment and medium, and the technical solution is as follows:

[0006] According to one aspect of the present application, a sample purification method is provided, the method comprising:

[0007] Obtain an adversarial sample, where the adversarial sample is a sample generated after performing an adversarial attack on a clean sample;

[0008] Adding a first noise to the adversarial image to obtain a noisy image, wherein the noisy image is an image obtained by performing a first disturbance on the adversarial image;

[0009] Calculating a noisy cross-modal attention corresponding to the noisy image, where the noisy cross-modal attention is used to describe the attention distribution of the noisy image and the text;

[0010] A second noise is added to the adversarial image based on the noisy cross-modal attention.

[0011] According to one aspect of the present application, a sample purification device is provided, the device comprising:

[0012] An acquisition module, used to acquire adversarial samples, wherein the adversarial samples are samples generated after performing adversarial attacks on clean samples, and the adversarial samples include adversarial images and texts;

[0013] An adding module, configured to add a first noise to the adversarial image to obtain a noisy image;

[0014] A calculation module, used to calculate the noisy cross-modal attention corresponding to the noisy image, where the noisy cross-modal attention is used to describe the attention distribution of the noisy image and the text;

[0015] A control module is used to add a second noise to the adversarial image based on the noisy cross-modal attention to obtain a purified image.

[0016] According to another aspect of the present application, a computer device is provided, which includes: a processor and a memory, wherein at least one computer program is stored in the memory, and the at least one computer program is loaded and executed by the processor to implement the sample purification method described above.

[0017] According to another aspect of the present application, a computer storage medium is provided, in which at least one computer program is stored. The at least one computer program is loaded and executed by a processor to implement the sample purification method described above.

[0018] According to another aspect of the present application, a computer program product is provided, which includes a computer program stored in a computer-readable storage medium; the computer program is read and executed from the computer-readable storage medium by a processor of a computer device, so that the computer device performs the sample purification method described above.

[0019] The beneficial effects of the technical solution provided by this application include at least:

[0020] The visual language model will produce incorrect predictions based on adversarial samples. A purified image is obtained by performing a purification operation on the adversarial image in the adversarial sample, and then a purified sample can be obtained. Specifically, a noisy image is obtained by performing random noise processing on the adversarial image. Since the noisy cross-modal attention of the noisy image is very close to the cross-modal attention corresponding to the clean image, the noisy cross-modal attention of the noisy image can be used as the optimization target, and random noise is added to the adversarial image, so that the adversarial cross-modal attention of the adversarial image is close to the noisy cross-modal attention of the noisy image, so as to complete the purification of the adversarial image and obtain a purified image. The present application is guided by noisy cross-modal attention. By adding noise to the adversarial image to achieve purification of the adversarial image, the visual language model can obtain correct prediction results based on the purified image. This purification method enhances the robustness of the visual language model.

[0021] Furthermore, the method of adding random noise to the adversarial image by using noisy cross-modal attention as the estimation direction not only optimizes the adversarial image, but also optimizes the cross-modal attention itself. This optimization enables the attention mechanism to focus on the key information in the image more accurately, thereby improving the accuracy of the prediction results output by the visual language model. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0023] Figure 1 is a schematic diagram of the architecture of a computer system provided by an exemplary embodiment of the present application;

[0024] Figure 2 is a schematic diagram of a sample purification method provided by an exemplary embodiment of the present application;

[0025] Figure 3 is a flow chart of a sample purification method provided by an exemplary embodiment of the present application;

[0026] Figure 4 is a flow chart of a sample purification method provided by an exemplary embodiment of the present application;

[0027] Figure 5 is a flow chart of a sample purification method provided by an exemplary embodiment of the present application;

[0028] Figure 6 is a flow chart of a sample purification method provided by an exemplary embodiment of the present application;

[0029] Figure 7 is a structural block diagram of a sample purification device provided by an exemplary embodiment of the present application;

[0030] Figure 8 It is a structural diagram of a computer device provided by an exemplary embodiment of the present application. DETAILED DESCRIPTION

[0031] In order to make the objectives, technical solutions and advantages of the present application clearer, the implementation methods of the present application will be further described in detail below with reference to the accompanying drawings.

[0032] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present application. Instead, they are merely examples of devices and methods consistent with some aspects of the present application as detailed in the appended claims.

[0033] The terms used in this application are for the purpose of describing specific embodiments only and are not intended to limit this application. The singular forms of "a", "said" and "the" used in this application and the appended claims are also intended to include plural forms unless the context clearly indicates other meanings. It should also be understood that the term "and / or" used herein refers to and includes any or all possible combinations of one or more associated listed items.

[0034] It should be understood that although the terms first, second, etc. may be used in the present disclosure to describe various information, such information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of the present disclosure, the first parameter may also be referred to as the second parameter, and similarly, the second parameter may also be referred to as the first parameter. Depending on the context, the word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining".

[0035] It should be noted that the data involved in this application (including but not limited to data used for analysis, training, recommendation, etc.) are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of relevant countries and regions. For example, the information such as setting operations involved in this application is obtained with full authorization.

[0036] Before introducing the technical solution of the present application, some terms involved in the present application are explained. The following related explanations can be combined arbitrarily with the technical solution of the embodiment of the present application as optional solutions, and they all belong to the protection scope of the embodiment of the present application. The embodiment of the present application includes at least part of the following contents.

[0037] Visual Language Model: A visual language model is a multimodal model that combines vision (image) and language (text), which can receive image and text input and generate task output. The visual language model can understand and process the association between images and text, and is used to handle a variety of tasks, such as image captioning, visual question answering (VQA), image-text matching, etc.

[0038] Clean samples: Clean samples refer to original samples without any modification or disturbance. Clean samples are standard samples in an ideal state and serve as a benchmark in the evaluation of visual language models. Clean samples include clean images and text.

[0039] Adversarial samples: Adversarial samples are samples that make minor modifications or perturbations to clean samples, causing the visual language model to make incorrect predictions. The perturbations in adversarial samples are not easily noticeable, but will cause the visual language model to make incorrect predictions or outputs. Among them, adversarial samples include adversarial images and text. The purification operation for adversarial samples in the embodiments of the present application can be regarded as a purification operation for adversarial images.

[0040] Noisy samples: adversarial samples with random noise added to the original adversarial samples. The range of random noise addition is limited or constrained by the infinite norm. The purpose of random noisy adversarial samples is to keep the adversarial nature while making the cross-modal attention of the noisy samples close to the cross-modal attention of the clean samples. The noisy samples include noisy images and texts.

[0041] Purified samples: refers to samples that have been purified from minor modifications or disturbances in adversarial samples. Purified samples can increase the probability that the visual language model will generate correct predictions. The purpose of purifying samples is to reduce adversarial disturbances that cause incorrect predictions of the visual language model and increase the probability that the visual language model will generate correct predictions. Among them, purified samples include purified images and text. In the embodiment of the present application, the purification operation on the adversarial sample obtains the purified sample, which can be understood as the purification operation on the adversarial image obtains the purified image.

[0042] Visual token: A visual token is a feature representation extracted from an image, which is obtained by encoding it through a visual encoder in a visual language model. The visual encoder divides the input image into several regions, each of which can be encoded as an image feature (or visual feature), and several regions can be encoded as several image features. The language alignment module in the visual language model aligns several image features to obtain multiple visual tokens, and each visual token can be a fusion representation of several image features. For example, one region of an image corresponds to one image feature, and one visual token can be a feature representation after the fusion of several image features. Exemplarily, the visual encoder in the visual language model divides the image into 16×16=256 regions and extracts 256 image features, which are processed into 32 visual tokens by the language alignment module, and each visual token can be a fusion representation of 256 image features. Optionally, different visual tokens have different focuses on the fusion of image features. For example, the first visual token focuses on the background area in the image, that is, the first visual token mainly represents the image features of the background area; the second visual token focuses on the object area in the image, that is, the second visual token mainly represents the image features of the object area.

[0043] Language tokens: Language tokens are feature representations extracted from text, encoded by the language model tokenizer in the visual language model. Optionally, the language model tokenizer divides the input text into several words (or subwords), and then converts each word into a corresponding feature vector, which can be regarded as language tokens of the text. Optionally, the text can usually be divided into several subwords, and the feature vector of each subword can be represented as a language token, and a language token usually corresponds to a subword in the text. Alternatively, the feature vectors of multiple subwords can be represented as a language token, and a language token usually corresponds to multiple subwords in the text. For example, a language token is the text feature of all subwords in the text.

[0044] Cross-modal attention: refers to an attention mechanism used by visual language models when processing multimodal data. Cross-modal attention refers to how one modality (such as text) allocates attention or focuses on the content of another modality (such as images) in multimodal data processing. In this application example, cross-modal attention refers to how the visual language model focuses on the relevant parts of the image in the sample when generating prediction results.

[0045] For example, in the application scenario of visual question answering, when processing images, the visual language model can focus on the image area related to the descriptive text or question based on cross-modal attention. For example, the task of the visual language model is to answer questions in images. The image input of the visual language model is a red car driving on the highway, and the text input of the visual language model is a question "What is the color of the car?" When the visual language model generates the first word "red" of the answer, it calculates the attention weights of all visual tokens (preset image features) in the image. The attention weights represent the importance of each visual token to the currently generated word, that is, the visual language model evaluates the extent to which each visual token affects the generation of the first word "red". Optionally, the visual language model may focus on the visual tokens that represent the car part in the image.

[0046] It should be noted that in the embodiments of the present application, purification operations are mainly performed on the adversarial images in the adversarial samples to obtain purified samples with purified images. Unless otherwise specified, the cross-modal attention corresponding to the clean samples can be equivalent to the cross-modal attention corresponding to the clean images; the cross-modal attention corresponding to the adversarial samples can be equivalent to the cross-modal attention corresponding to the adversarial images; the cross-modal attention corresponding to the noisy samples can be equivalent to the cross-modal attention corresponding to the noisy images; the cross-modal attention corresponding to the purified samples can be equivalent to the cross-modal attention corresponding to the purified images.

[0047] Figure 1 The schematic diagram of the architecture of a computer system provided by an embodiment of the present application is shown. The computer system may include: a terminal 100 and a server 200.

[0048] The terminal 100 may be an electronic device such as a mobile phone, a tablet computer, a vehicle-mounted terminal (vehicle computer), a wearable device, a personal computer (PC), a vehicle-mounted terminal, etc. The terminal 100 may be installed with a client that runs a target application, and the target application may be an application that supports sample purification, which is not limited in this application. In addition, this application does not limit the form of the target application, including but not limited to an application (Application, App), a mini-program, a client, etc. installed in the terminal 100, and may also be in the form of a web page.

[0049] The server 200 may be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server providing cloud computing services, cloud database, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (CDN), and cloud servers providing basic cloud computing services such as big data and artificial palm image recognition platforms. The server 200 may be a backend server of the above-mentioned target application, used to provide backend services for the client of the target application.

[0050] Among them, cloud technology refers to a hosting technology that unifies hardware, software, network and other resources in a wide area network or local area network to realize data computing, storage, processing and sharing. Cloud technology is a general term for network technology, information technology, integration technology, management platform technology, application technology, etc. based on the cloud computing business model. It can form a resource pool, which is used on demand and flexible and convenient. Cloud computing technology will become an important support. The background services of the technical network system require a large amount of computing and storage resources, such as video websites, picture websites and more portal websites. With the high development and application of the Internet industry, in the future, each item may have its own identification mark, and all need to be transmitted to the background system for logical processing. Data of different levels will be processed separately. All kinds of industry data require strong system backing support, which can only be achieved through cloud computing.

[0051] In some embodiments, the above-mentioned server can also be implemented as a node in a blockchain system. Blockchain is a new application model of computer technologies such as distributed data storage, peer-to-peer transmission, consensus mechanism, and encryption algorithm. Blockchain is essentially a decentralized database, a string of data blocks generated by cryptographic methods. Each data block contains a batch of network transaction information, which is used to verify the validity of its information (anti-counterfeiting) and generate the next block. Blockchain can include the blockchain underlying platform, platform product service layer, and application service layer.

[0052] The terminal 100 and the server 200 may communicate with each other via a network, such as a wired or wireless network.

[0053] In the sample purification method provided in the embodiment of the present application, the execution subject of each step may be a computer device, which refers to an electronic device with data calculation, processing and storage capabilities. Figure 1Taking the computer system shown as an example, the sample purification method can be executed by the terminal 100, or by the server 200, or by the terminal 100 and the server 200 interacting and cooperating to execute the sample purification method, which is not limited in this application.

[0054] For example, Figure 2 A schematic diagram of a sample purification method provided by an exemplary embodiment of the present application is shown. The method is executed by a computer device, which may be Figure 1 The terminal 100 and / or the server 200 shown in the figure. In this embodiment, the computer device is taken as an example of the server 200. The server 200 executes the sample purification method.

[0055] In the related art, some preprocessing operations are performed on the image input into the visual language model to achieve the purification of adversarial samples. For example, the image is smoothed, that is, a smoothing filter is applied to reduce image noise, thereby reducing the impact of adversarial attacks; for another example, the image is scaled, that is, the image features are changed by changing the scale of the image (such as zooming), thereby reducing the effectiveness of adversarial attacks. However, the purification of adversarial samples in the related art mainly relies on traditional image processing technology, resulting in unsatisfactory purification effects for adversarial samples. In practical applications, adversarial samples will still have an impact on the task output of the visual language model.

[0056] In order to solve the above problems, the present application proposes a sample purification method. In order to reduce adversarial attacks in adversarial samples and enable the visual language model to produce correct prediction results, the adversarial samples (mainly adversarial images in adversarial samples) input into the visual language model can be purified, and purified samples (purified images) are obtained after the purification process. The embodiment of the present application takes cross-modal attention as the target direction of the purification process to guide the adversarial sample to complete the purification operation. Optionally, by adding random noise to the adversarial sample, the adversarial cross-modal attention of the adversarial sample is guided to be close to the clean cross-modal attention of the clean sample, so that the visual language model produces correct prediction results.

[0057] In the application scenario of visual question answering, the visual language model can answer natural language questions about images. Figure 2 For illustration, the clean sample 10 includes a clean image and text; the adversarial sample 20 includes an adversarial image and text; the noisy sample 30 includes a noisy image and text; and the cleaned sample 40 includes a cleaned image and text. The texts of the clean sample 10, the adversarial sample 20, the noisy sample 30, and the cleaned sample 40 are descriptions of a natural language question "What is this man doing?".

[0058] Optionally, the clean sample 10 is input into the visual language model 50, and an output result 60 can be obtained, that is, the correct answer "hit the ball". When generating the answer "hit the ball", the visual language model 50 calculates the clean cross-modal attention of the clean image in the clean sample 10. Optionally, the visual language model 50 calculates the attention weights of all visual tokens in the clean image. The visual language model 50 divides the clean image into several regions, and the image features of each region (or the image features corresponding to multiple regions) can be represented as a visual token. The attention weight represents the importance of each visual token to the answer "hit the ball", that is, the visual language model will evaluate the extent to which each visual token affects the generation of "hit the ball". Optionally, the cross-modal attention map 70 is a visual representation of the cross-modal attention, which shows the attention paid by the visual language model to different visual tokens in the image when processing the task.

[0059] In some embodiments, the cross-modal attention of the clean sample 10 and the adversarial sample 20 is significantly different, which can also be understood as the cross-modal attention of the clean image and the adversarial image is significantly different. The adversarial cross-modal attention of the adversarial sample 20 can be guided to approach the clean cross-modal attention of the clean sample 10, thereby achieving the purification of the adversarial sample 20. However, in the scenario of adversarial purification, it is often impossible to directly obtain the clean sample 10, and then it is impossible to directly obtain the clean cross-modal attention of the clean sample 10.

[0060] In a possible implementation, a noisy sample 30 is obtained by adding a first noise to the adversarial image of the adversarial sample 20. Compared with the adversarial cross-modal attention of the adversarial sample 20, the noisy cross-modal attention of the noisy sample 30 is closer to the clean cross-modal attention corresponding to the clean sample 10 (as shown by the attention weight distribution in the cross-modal attention map corresponding to the noisy sample 30 being close to the attention weight distribution in the cross-modal attention map corresponding to the clean sample 10). Therefore, the noisy cross-modal attention of the noisy sample 30 is used as a reference estimation direction to guide the adversarial cross-modal attention of the adversarial sample 20 to be optimized, which can be considered as a purification operation for the adversarial sample 20 to make it closer to the clean sample 10. Optionally, the purification of the adversarial sample 20 is achieved by guiding the adversarial cross-modal attention corresponding to the adversarial sample 20 to be close to the noisy cross-modal attention corresponding to the noisy sample 30.

[0061] In some embodiments, the noisy cross-modal attention corresponding to the noisy sample 30 is taken as a target, and the adversarial cross-modal attention corresponding to the adversarial sample 20 is guided to approach the noisy cross-modal attention corresponding to the noisy sample 30. Optionally, with the goal of minimizing the attention difference value as the optimization goal, a second noise is added to the adversarial image to generate a purified image, that is, a purification operation is performed on the adversarial sample 20 and a purified sample 40 is generated. Among them, the attention difference value is the difference between the noisy cross-modal attention corresponding to the noisy sample 30 and the adversarial cross-modal attention corresponding to the adversarial sample 20. The visual language model 50 can generate the correct answer "hit" based on the purified sample 40.

[0062] In some embodiments, the purified sample 40 is obtained based on the following steps: first, the norm of the attention difference value is calculated, and the norm of the attention difference value is used to measure the distance between the adversarial cross-modal attention corresponding to the adversarial sample 20 and the noisy cross-modal attention corresponding to the noisy sample 30. Secondly, the gradient of the attention difference value for the pixel in the adversarial image is calculated, and the gradient is used to indicate how to add the second noise to minimize the attention difference value under the current adversarial sample 20. Then, the gradient direction of the gradient is determined based on the sign function, and the gradient direction is used to indicate the direction in which the second noise is added to the adversarial image in the adversarial sample 20, that is, the second noise is added to the pixels in the adversarial image, and the pixels in the adversarial image are adjusted in the gradient direction to reduce the attention difference value. Optionally, the gradient is used as the input of the sign function. When the gradient is positive, the output of the sign function is 1, that is, the adversarial sample 20 is controlled to increase the pixel value in the adversarial image in the direction of the gradient; when the gradient is negative, the output of the sign function is -1, that is, the adversarial sample 20 is controlled to reduce the pixel value in the adversarial image in the direction of the gradient; when the gradient is zero, the output of the sign function is 0, that is, the adversarial sample is controlled to keep the pixel value in the adversarial image unchanged in the direction of the gradient (basically the pixel value will change, and there is almost no unchanged situation). Optionally, the pixel value in the adversarial image is increased or decreased in the direction of the gradient to obtain a purified image after adding the second noise.

[0063] In some embodiments, the size of the second noise added to the adversarial image is controlled. Optionally, the gradient contains not only information about the gradient direction, but also information about the intensity. Among them, the intensity information of the gradient is positively correlated with the size of the added second noise. Optionally, the adversarial image in the adversarial sample 20 is divided by pixel granularity, and second noises of different perturbation degrees (different sizes) are added to different pixel positions of the adversarial image based on the gradient direction. Optionally, pixels with larger gradients in the adversarial image should be subjected to greater perturbations, that is, relatively larger noises are added; pixels with smaller gradients in the adversarial image should be subjected to smaller perturbations, that is, relatively smaller noises are added. Optionally, the maximum value and the minimum value of the gradient are obtained, and the standard value of the gradient is obtained based on the maximum value and the minimum value of the gradient. The standard value of the gradient is the gradient value after the gradient is normalized, and the gradient is normalized to [0, 1]. In some embodiments, due to the large difference in the standard value of the gradient, the standard value of the gradient can be expressed based on the average value of the standard value of the gradient. The perturbation amplitude of the second noise is determined based on the standard value of the gradient and the average value of the standard value of the gradient. The perturbation amplitude includes a perturbation factor for determining the size of the second noise, and the size of the second noise added to each pixel in the adversarial image is determined based on the perturbation factor.

[0064] In some embodiments, the purified image is obtained by the above purification operation on the adversarial image, that is, the purified sample 40 is obtained by the purification operation on the adversarial sample 20. Optionally, the purified sample 40 is input into the visual language model 50, and the visual language model 50 can obtain the correct output result "hit" based on the purified sample 40.

[0065] Figure 3 is a flow chart of a sample purification method provided by an exemplary embodiment of the present application. The method can be executed by a computer device, which can be Figure 1 The terminal 100 or the server 200 in the embodiment. The method comprises the following steps.

[0066] Step 210: Obtain adversarial samples;

[0067] The adversarial sample is a sample generated after performing an adversarial attack on a clean sample. Optionally, the clean sample includes a clean image and text. The adversarial sample includes an adversarial image and text.

[0068] In some embodiments, a clean sample refers to an original sample without any modification or perturbation. An adversarial sample is a sample generated after performing an adversarial attack on a clean sample. The adversarial attack includes making slight modifications or perturbations to the clean sample. The modifications or perturbations in the adversarial sample are not easily noticeable, but may cause the visual language model to make incorrect predictions and generate incorrect prediction results (output results).

[0069] In some embodiments, adversarial samples can be used to evaluate the performance of visual language models in the face of small perturbations. By introducing adversarial samples, the robustness of visual language models in practical applications can be tested to see whether they are vulnerable to attacks or misleading. Adversarial samples may be used as attack tools to try to gain competitive advantages or conduct malicious activities by interfering with visual language models. Therefore, the purification of adversarial samples is particularly important.

[0070] For example, there is a clean image showing a cat, and in the visual question answering scenario, the visual language model correctly classifies it as a "cat". By applying an adversarial attack to the clean image, such as an adversarial attack based on the FGSM (Fast Gradient Sign Method) method, an adversarial image can be generated. The generated adversarial image still looks like a cat, but the visual language model may incorrectly classify it as a "dog" or other categories.

[0071] In some embodiments, in order to reduce adversarial attacks in adversarial samples and enable the visual language model to produce correct prediction results, a purification operation is performed on the adversarial image in the adversarial sample to obtain a purified image.

[0072] Step 220: Adding a first noise to the adversarial image to obtain a noisy image;

[0073] The noisy image is an image after the first disturbance is performed on the adversarial image. The first noise refers to the random noise added to the adversarial image. The first noise is generated randomly or according to a certain rule, and the purpose is to disturb the pixel value of the adversarial image. Optionally, the first noise can be Gaussian noise, uniform noise or other types of random noise.

[0074] In some embodiments, the visual language model is a multimodal model that combines vision (image) and language (text), which can receive image and text input and generate task output. The visual language model can understand and process the association between images and text, and is used to process a variety of tasks. Optionally, the visual language model includes at least one of a deep learning neural network model (Deep Neural Network, DNN), a convolutional neural network model (Convolutional Neural Network, CNN), a Transformer model, a contrastive language-image pre-training model (Contrastive Language-Image Pre-Training, CLIP), an extreme learning machine model (Extreme Learning Machine, ELM) or other neural network models, but is not limited to this, and the embodiments of the present application are not specifically limited to this.

[0075] In some embodiments, a first noise is added to the adversarial image to obtain a noisy image. Optionally, the first noise added to the adversarial image is uniform noise. The noisy image can be expressed as:

[0076]

[0077] in, represents a noisy image, represents the adversarial image, Represents the first noise, that is, the first noise is based on the first distribution interval The first noise value obtained by pseudo-random sampling is determined, and the first distribution interval is the range of uniform distribution. Indicates that in the adversarial image Adding the infinite norm restriction on of random noise.

[0078] In one possible implementation, the addition of the first noise is implemented by a computer program. Optionally, a program for adding noise is installed in a computer device, and a uniformly distributed first noise can be generated using a programming language (such as NumPy in Python, OpenCV library, etc.), and the first noise is added to the adversarial image. Optionally, a program for adding noise is installed in a computer device, and the program uses the OpenCV library to read the adversarial image to which the first noise is to be added, and uses a random number generator to generate uniformly distributed random numbers, which will be used as the noise value of the first noise. Optionally, the value of the generated random noise (first noise) is adjusted according to the required noise intensity. Among them, the noise intensity can be controlled by adjusting the range of the random number generator. Then, the noise value of the first noise generated by the random number generator is added to the pixels of the adversarial image.

[0079] In one possible implementation, the addition of the first noise is achieved through a noise model, for example, using a generative adversarial network (GANs), a variational autoencoder (VAEs) or other deep learning models to generate the first noise, and then adding the first noise to the adversarial image. Exemplarily, the example of adding a first noise to an adversarial image by a generative adversarial network to generate a noisy image is used for illustration. The generative adversarial network receives a random noise (first noise) as input. The random noise can be sampled from a uniform distribution, and the generative adversarial network outputs a uniformly distributed noise image based on the random noise, and then the generative adversarial network performs noise addition processing, and adds the generated noise image to the adversarial image, that is, superimposing the pixel values ​​of the noise image on the adversarial image to obtain the noisy image. The above is only an exemplary illustration, and the present application does not limit the method of adding noise.

[0080] Step 230: Calculate the noisy cross-modal attention corresponding to the noisy image;

[0081] Among them, noisy cross-modal attention is used to describe the attention distribution of noisy images and texts.

[0082] In some embodiments, the cross-modal attention of clean samples and adversarial samples is significantly different, which can also be understood as the clean cross-modal attention of clean images and the adversarial cross-modal attention of adversarial images are significantly different. The adversarial image can be purified by guiding the adversarial cross-modal attention of the adversarial image to approach the clean cross-modal attention of the clean image. However, in the scenario of adversarial purification, it is often impossible to directly obtain a clean image, and then it is impossible to directly obtain the clean cross-modal attention of the clean image.

[0083] In one possible implementation, a noisy image is obtained by adding a first noise to the adversarial image. Compared with the adversarial cross-modal attention of the adversarial image, the noisy cross-modal attention of the noisy image is closer to the clean cross-modal attention of the clean image. Therefore, taking the noisy cross-modal attention of the noisy image as a reference estimation direction, guiding the adversarial cross-modal attention of the adversarial image to optimize can be considered as a purification operation for the adversarial image to make it closer to the clean image. Optionally, the adversarial image is purified by guiding the adversarial cross-modal attention of the adversarial image to be close to the noisy cross-modal attention of the noisy image.

[0084] In some embodiments, the noisy cross-modal attention corresponding to the noisy image is calculated. Cross-modal attention refers to an attention mechanism of the visual language model when processing multimodal data. Noisy cross-modal attention refers to how the visual language model pays attention to the attention size (or attention weight) of the noisy image in the noisy sample when generating a prediction result based on the noisy sample.

[0085] Optionally, the visual language model calculates the attention weights of all visual tokens in the noisy image. The visual language model divides the noisy image into several regions. The image feature corresponding to each region can be represented as a visual token (or multiple image features corresponding to multiple regions can be represented as a visual token). The attention weight represents the importance of each visual token to the visual language model generating the prediction result, that is, the visual language model will evaluate the extent to which each visual token affects the generation of the prediction result.

[0086] Exemplarily, the task of the visual language model is to answer questions in a noisy image. The noisy image input of the visual language model is a car driving on a highway, and the text input of the visual language model is a question "What is the color of the car?" When the visual language model generates a prediction result, it calculates the attention weights of all visual tokens in the noisy image. The attention weight indicates the importance of each visual token to the current prediction result, that is, the visual language model evaluates the extent to which each visual token affects the generation of the prediction result. Optionally, the visual language model may focus on the visual tokens that represent the car part in the noisy image.

[0087] Step 240: Add a second noise to the adversarial image based on the noisy cross-modal attention to obtain a purified image.

[0088] The purified image is an image after the second disturbance is performed on the adversarial image. The second noise refers to the random noise added to the adversarial image. The second noise is generated randomly or according to a certain rule, and the purpose is to disturb the pixel value of the adversarial image. Optionally, the second noise can be Gaussian noise, uniform noise or other types of random noise.

[0089] In some embodiments, the noisy cross-modal attention of the noisy image is taken as a target to guide the adversarial cross-modal attention of the adversarial image to approach the noisy cross-modal attention corresponding to the noisy image.

[0090] Optionally, by continuing to add random noise to the adversarial image, the adversarial cross-modal attention of the adversarial image can be guided to approach the noisy cross-modal attention corresponding to the noisy image, thereby generating a purified image. Optionally, the random noise added to the adversarial image is a noise value randomly sampled in an interval range of a range distribution. With minimizing the difference between the adversarial cross-modal attention and the noisy cross-modal attention as the optimization goal, a second noise is added to the adversarial image to generate a purified image.

[0091] In some embodiments, a second noise is added to the adversarial image based on the noisy cross-modal attention to obtain a purified image. Optionally, the visual language model can obtain a correct prediction result based on the purified image.

[0092] Taking the clean image as a reference, the degree of perturbation of the purified image compared to the clean image may be greater than that of the adversarial image compared to the clean image. The visual language model obtains incorrect prediction results based on the adversarial image, but obtains correct results based on the purified image. Although the purified image may be visually noisier, the purified cross-modal attention of the purified image may be closer to the clean cross-modal attention of the clean image, because the second noise added to the purified image adjusts the attention distribution of the visual language model and enhances the visual language model's attention to the features in the purified image that are related to the prediction task. Therefore, the prediction results of the visual language model for the purified image are closer to the prediction results for the clean image.

[0093] In summary, the method provided in this embodiment is that the visual language model will produce incorrect predictions based on adversarial samples. A purified image is obtained by performing a purification operation on the adversarial image in the adversarial sample, and then a purified sample can be obtained. Specifically, a noisy image is obtained by randomly adding noise to the adversarial image. Since the noisy cross-modal attention of the noisy image is very close to the cross-modal attention corresponding to the clean image, the noisy cross-modal attention of the noisy image is used as the optimization target, and random noise is added to the adversarial image, so that the adversarial cross-modal attention of the adversarial image is close to the noisy cross-modal attention of the noisy image, so as to complete the purification of the adversarial image and obtain the purified image. Furthermore, the present application is guided by cross-modal attention, and by adding noise to the adversarial sample to achieve the purification of the adversarial image, the visual language model can obtain correct predictions based on the purified image (or purified sample). This purification method enhances the robustness of the visual language model. Furthermore, the method of adding noise to adversarial images guided by cross-modal attention as an estimation method is not only for processing adversarial images, but also for optimizing the cross-modal attention mechanism itself. This optimization enables the attention mechanism to focus more accurately on the key information in the image, thereby improving the accuracy of the prediction results output by the visual language model.

[0094] Calculate the cross-modal attention corresponding to the noisy image

[0095] In some embodiments, the noisy cross-modal attention of the noisy image is closer to the clean cross-modal attention of the clean image than the adversarial cross-modal attention of the adversarial image. Optionally, the adversarial image is purified by guiding the adversarial cross-modal attention of the adversarial image to be close to the noisy cross-modal attention of the noisy image.

[0096] Figure 4 A flow chart of a sample purification method provided by an exemplary embodiment of the present application is shown, wherein the above step 230 can be replaced by step 231 , step 232 and step 233 .

[0097] Step 231: Encode the noisy image into image features based on a visual encoder, and align the image features through a visual language model to obtain at least one visual token;

[0098] Each visual token in the at least one visual token is used to indicate an image feature of the noisy image.

[0099] In some embodiments, the visual language model includes a visual encoder, which is used to extract feature vectors from the noisy image and encode the noisy image into image features (or visual features). Optionally, the visual encoder receives the noisy image as input and divides the noisy image into several regions, each of which can be the same size, and then extracts features from the image blocks in each region, and the image blocks in each region can be encoded as an image feature. Optionally, multiple visual tokens are obtained by aligning multiple image features through a language alignment module in the visual language model, and each visual token can be a fusion representation of several image features. For example, one region of the noisy image corresponds to one image feature, and one visual token can be a feature representation after the fusion of several image features. Exemplarily, the visual encoder in the visual language model divides the noisy image into 16*16=256 regions and extracts 256 image features, which are processed into 32 visual tokens by the language alignment module, and each visual token can be a fusion representation of 256 image features.

[0100] In one possible implementation, an image feature can be regarded as a visual token of a noisy image, and each visual token is used to indicate the image feature of a region of the noisy image. Alternatively, multiple image features can be regarded as a visual token of a noisy image, and each visual token is used to indicate the image features of multiple regions of the noisy image. For example, the noisy image is divided into 16×16 regions, each region corresponds to an image feature, and the visual language model processes the noisy image to obtain 32 visual tokens, each of which can be a fusion representation of the image features of all regions.

[0101] The following example uses a visual token to indicate the image features of a region of an image.

[0102] Exemplarily, assume a noisy image showing a man hitting a ball on the grass. The noisy image is input into a visual language model. The visual encoder in the visual language model divides the noisy image into several fixed-size regions, for example, the noisy image is divided into 32 regions. Optionally, for each region, the visual encoder extracts the features of the region to obtain image features. The image features of each region are encoded as a visual token, and each visual token represents the visual information of the corresponding region in the noisy image. For example, region 1 represents the grass in the noisy image, and visual token 1 is used to indicate the image features of the grass part in the noisy image; region 2 represents the grass and the man's feet in the noisy image, and visual token 2 is used to indicate the image features of the grass and the man's feet in the noisy image; region 3 represents the man's head in the noisy image, and visual token 3 is used to indicate the image features of the man's head part in the noisy image; region 4 represents the man's arms and the club in the noisy image, and visual token 4 is used to indicate the image features of the man's arms and the club part in the noisy image; region 5 represents the ball in the noisy image, and visual token 5 is used to indicate the image features of the ball part in the noisy image; region 6 represents the bottom of the club and the grass in the noisy image, and visual token 6 is used to indicate the image features of the bottom of the club and the grass in the noisy image... and so on.

[0103] Step 232: Encode the text into text features based on the language model tokenizer, and align the text features through the visual language model to obtain at least one language token;

[0104] Each of the at least one language token is used to indicate a text feature of the text.

[0105] In some embodiments, the visual language model includes a language model word segmenter, which is used to extract feature vectors from text and encode the text into text features. Optionally, the language model word segmenter receives text as input and decomposes the input text into smaller text units. Optionally, the text units include words, subwords or strings. Take the decomposition of text into several subwords as an example: the language model word segmenter decomposes the input text into several subwords, and then extracts features for each subword, and each subword can be encoded as a corresponding feature vector. Optionally, the visual language model can also retain the order information of each subword in the text, add a position code to the feature vector corresponding to each subword, and the position code is a value calculated based on the position of the subword in the text. The language alignment module in the visual language model aligns the feature vector corresponding to each subword with the position code to obtain a language token. Among them, the feature vector corresponding to a subword can be regarded as a language token of the text. Or, the feature vectors corresponding to multiple subwords can be regarded as a language token of the text. Alternatively, the text can be generally divided into several subwords, and the feature vector of each subword can be represented as a language token, and a language token generally corresponds to a subword in the text. Alternatively, the feature vectors of multiple subwords can be represented as a language token, and a language token generally corresponds to multiple subwords in the text, for example, a language token is a text feature of all subwords in the text.

[0106] The following example uses a language token to indicate the text feature of a subword of a text.

[0107] Exemplarily, assume that the text for the above-mentioned noisy image is a question description, and the question description indicated by the text is "What is this man doing?". The text is input into the visual language model, and the language model tokenizer in the visual language model decomposes the text into several subwords, for example, decomposing "What is this man doing?" into the following subwords: ["this", "man", "in", "doing", "what", "?"]. For each subword, the language model tokenizer extracts the corresponding feature vector. In order to retain the order information of the subwords in the text, the feature vector of each subword may be added with a position code. The position code is a value calculated according to the position of the subword in the text. The language alignment module in the visual language model aligns the feature vector of each subword with the position code to generate a language token. Each language token represents the text feature of the subword in the corresponding area in the text. For example, subword 1 represents "this" in the text, and language token 1 is used to indicate the text features of the subword "this"; subword 2 represents "man" in the text, and language token 2 is used to indicate the text features of the subword "man"; subword 3 represents "in" in the text, and language token 3 is used to indicate the text features of the subword "in"; subword 4 represents "do" in the text, and language token 4 is used to indicate the text features of the subword "do"; subword 5 represents "what" in the text, and language token 5 is used to indicate the text features of the subword "what"... and so on.

[0108] Step 233: The noisy cross-modal attention when generating the noisy prediction result is calculated by the visual language model.

[0109] Among them, the noisy prediction result is the prediction result of the visual language model after fusing multiple visual tokens and multiple language tokens.

[0110] In some embodiments, all visual tokens in the noisy image and their corresponding attention weights are integrated together to form an overall representation of the noisy image. The visual language model generates a noisy prediction result corresponding to the noisy image based on the description of the integrated visual tokens and the corresponding language tokens. The noisy prediction result is the prediction result of the visual language model for the integrated visual tokens (noisy image) based on the language tokens (the problem description corresponding to the text).

[0111] In some embodiments, when the visual language model generates a noisy prediction result based on the noisy image, the attention weights of all visual tokens in the noisy image are calculated, and the attention weights represent the importance of each visual token to the noisy prediction result, that is, the visual language model evaluates the extent to which each visual token affects the generation of the noisy prediction result. Optionally, when generating the noisy prediction result, the visual language model focuses on the visual tokens in the noisy image whose attention weights are greater than a first preset weight threshold, and ignores the visual tokens in the noisy image whose attention weights are less than a second preset weight threshold.

[0112] In some embodiments, the noisy cross-modal attention can be represented by a noisy cross-modal attention map, that is, the noisy cross-modal attention map is a visual representation of the noisy cross-modal attention. The noisy cross-modal attention map has several dimensions. For example, the noisy cross-modal attention map can represent the number of layers and the number of visual tokens of the visual language model. The noisy cross-modal attention map can more clearly represent the attention distribution of visual tokens in the noisy image. Optionally, in the noisy cross-modal attention map, the visual tokens in the noisy image whose attention weights are greater than the first preset weight threshold can be represented as a first form, the visual tokens in the noisy image whose attention weights are less than the second preset weight threshold can be represented as a second form, and the visual tokens in the noisy image whose attention weights are less than the first preset weight threshold and greater than the second preset weight threshold can be represented as a third form. Optionally, the visual salience of the first form is higher than that of the third form, and the visual salience of the third form is higher than that of the second form. This application is not limited to this.

[0113] For example, suppose a noisy image is input to the visual language model, and the noisy image shows a man hitting a ball on the grass. The text corresponding to the noisy image is a question description, and the question description indicated by the text is "What is this man doing?" For example, region 1 in the noisy image represents the grass; region 2 in the noisy image represents the grass and the man's feet; region 3 in the noisy image represents the man's head; region 4 in the noisy image represents the man's arms and the club; region 5 in the noisy image represents the ball; region 6 in the noisy image represents the sky, etc.; subword 1 of the text represents "this"; subword 2 of the text represents "man"; subword 3 of the text represents "in"; subword 4 of the text represents "do"; subword 5 of the text represents "what", etc. When the visual language model generates a noisy prediction result for the noisy image, it calculates the attention weight of each subword in the text to each region (visual token) in the noisy image, which reflects the degree of attention paid by the visual language model to the visual token when generating the noisy prediction result. Optionally, when generating a noisy prediction result, the visual language model may focus on areas with relatively high attention weights in the noisy image, for example, the attention weight of the visual token corresponding to area 1 in the noisy image is 0.05, the attention weight of the visual token corresponding to area 2 in the noisy image is 0.1, the attention weight of the visual token corresponding to area 3 in the noisy image is 0.1, the attention weight of the visual token corresponding to area 4 in the noisy image is 0.3, the attention weight of the visual token corresponding to area 5 in the noisy image is 0.2, the attention weight of the visual token corresponding to area 6 in the noisy image is 0.01, etc. For example, when generating a noisy prediction result, the visual language model will focus on the "man's arm and club" indicated by area 4 in the noisy image and the "ball" indicated by area 5, and ignore the "sky" indicated by area 6 in the noisy image. The visual language model combines the problem description corresponding to the text to obtain a noisy prediction result of "throwing the ball", "hitting the ball" or other results.

[0114] In summary, the method provided in the embodiment of the present application encodes the noisy image into several groups of feature vectors through a visual encoder, and each visual token represents the image features of one or more regions of the image (one or more groups of feature vectors). This decomposition can simplify the complex information of the noisy image into a set of structured features, which helps the visual language model to process and understand the different parts of the noisy image. The text is encoded into several groups of feature vectors through a language model word segmenter, and each language token represents one or more subwords of the text. This decomposition converts the text into structured features, which helps the visual language model understand the semantics of the text. The noisy cross-modal attention is calculated through the visual language model, allowing the visual language model to pay attention to the correlation between the noisy image and the text when generating the noisy prediction results, helping the visual language model to determine which image areas in the noisy image are most related to which subword in the text, so as to more accurately understand and interpret the relationship between the image and the text, thereby generating more accurate and relevant prediction results.

[0115] Add a second noise to the adversarial image with the goal of minimizing the attention difference

[0116] In some embodiments, the noisy cross-modal attention of the noisy image is used as a target to guide the adversarial cross-modal attention of the adversarial image to be close to the noisy cross-modal attention corresponding to the noisy image. Optionally, with the goal of minimizing the difference between the noisy cross-modal attention and the adversarial cross-modal attention, a second noise is added to the adversarial image to generate a purified image.

[0117] Figure 5 A flow chart of a sample purification method provided by an exemplary embodiment of the present application is shown, wherein the above step 240 can be replaced by step 241 .

[0118] Step 241: With the goal of minimizing the attention difference value, add a second noise to the adversarial image to obtain a purified image.

[0119] Among them, the attention difference value is the difference between the noisy cross-modal attention and the adversarial cross-modal attention.

[0120] In some embodiments, the noisy cross-modal attention is the cross-modal attention corresponding to the noisy image, which is used to describe the attention distribution of the noisy image and the text. The adversarial cross-modal attention is the cross-modal attention corresponding to the adversarial image, which is used to describe the attention distribution of the adversarial image and the text. Optionally, the noisy cross-modal attention corresponding to the noisy image is calculated based on the visual tokens and language tokens in the noisy image, and the adversarial cross-modal attention corresponding to the adversarial image is calculated based on the visual tokens and language tokens in the adversarial image.

[0121] It should be noted that the calculation of adversarial cross-modal attention and noisy cross-modal attention is implemented in the same way. The calculation of the adversarial cross-modal attention corresponding to the adversarial image can refer to the calculation of the noisy cross-modal attention in the above steps 231, 232 and 233, which will not be repeated here.

[0122] In some embodiments, an attention difference value can be obtained based on the adversarial cross-modal attention and the noisy adversarial cross-modal attention, where the attention difference value is the difference between the noisy cross-modal attention and the adversarial cross-modal attention. Optionally, a second noise is added to the adversarial image with the goal of minimizing the attention difference value.

[0123] Alternatively, the attention difference value can be expressed as:

[0124]

[0125] in, represents the attention difference value, represents the adversarial image, Represents a noisy image, that is, in the adversarial image Add the first noise The obtained noisy image is Represents text, represents the visual language model, represents the adversarial cross-modal attention corresponding to the adversarial image, represents the noisy cross-modal attention corresponding to the noisy image.

[0126] In some embodiments, the noisy adversarial cross-modal attention corresponding to the noisy image is close to the clean cross-modal attention corresponding to the clean image, so the noisy cross-modal attention can be considered as a reference point for purification. Optionally, the purification of the adversarial image is achieved by guiding the adversarial cross-modal attention to be close to the noisy cross-modal attention, that is, minimizing the difference (attention difference value) between the adversarial cross-modal attention and the noisy cross-modal attention. In one possible implementation method, the attention distribution (or attention weight) of the visual tokens in the adversarial cross-modal attention can be affected by adding a second noise to the adversarial image, thereby achieving purification of the adversarial image and obtaining a purified image.

[0127] Alternatively, the cleansed image can be represented as:

[0128]

[0129] in, represents the purified image, represents the adversarial image, represents a noisy image, Represents text, represents the visual language model, represents the second noise, that is, the second noise is based on the second distribution interval The second noise value obtained by pseudo-random sampling is determined, and the second distribution interval is the range of uniform distribution. represents the adversarial cross-modal attention corresponding to the adversarial image, represents the noisy cross-modal attention corresponding to the noisy image, represents the attention difference value, Represents the minimization of the attention difference value.

[0130] In some embodiments, with the goal of minimizing the attention difference value, a second noise is added to the adversarial image to obtain a purified image. Optionally, the second noise added to the adversarial image is uniform noise.

[0131] In some embodiments, the image after adding the second noise to the adversarial image is input into the visual language model for prediction. Optionally, based on the prediction result of the visual language model for the image after adding the second noise, it is determined whether the image is a purified image: if the prediction result of the visual language model for the image is a correct prediction result (the same as the prediction result of the clean image), the image can be determined to be a purified image; if the prediction result of the visual language model for the image is an incorrect prediction result (different from the prediction result of the clean image), random noise is continued to be added to the adversarial image based on the noisy cross-modal attention to adjust the attention distribution of the visual tokens in the adversarial image until the prediction result of the visual language model is a correct prediction result.

[0132] In one possible implementation, a preset question-and-answer score threshold is set in the visual language model, and whether the prediction result of the visual language model is a correct prediction result is determined based on the preset question-and-answer score threshold. Optionally, when the question-and-answer score of the visual language model for the image after adding the second noise to the adversarial image is greater than the preset question-and-answer score threshold, the prediction result of the visual language model is a correct prediction result, and the image at this time is a purified image; when the question-and-answer score of the visual language model for the image after adding the second noise to the adversarial image is less than the preset question-and-answer score threshold, the prediction result of the visual language model is an incorrect prediction result, then random noise continues to be added to the adversarial image based on the noisy cross-modal attention until the visual language model outputs a correct prediction result. This application does not limit the specific value of the preset question-and-answer score threshold.

[0133] In an embodiment of the present application, by calculating the adversarial cross-modal attention corresponding to the adversarial image and the noisy cross-modal attention corresponding to the noisy image, a second noise is added to the adversarial image with the goal of minimizing the difference (attention difference value) between the adversarial cross-modal attention and the noisy cross-modal attention. This guidance method with cross-modal attention as the estimation direction can obtain a purified image. This method not only processes the noise of the adversarial image, but also optimizes the cross-modal attention mechanism itself, so that the attention mechanism can more accurately focus on the key information in the adversarial image and ignore irrelevant or interfering information, thereby improving the accuracy and efficiency of cross-modal tasks.

[0134] Controls the direction of the noise added to the adversarial image

[0135] In some embodiments, with the goal of minimizing the difference between the adversarial cross-modal attention and the noisy adversarial cross-modal attention, a second noise is added to the adversarial image to generate a purified image. Optionally, the purified image is generated by controlling the noise direction of adding the second noise to the adversarial image.

[0136] Figure 6 A flow chart of a sample purification method provided by an exemplary embodiment of the present application is shown, wherein the above step 241 can be replaced by step 241a, step 241b and step 241c.

[0137] Step 241a: Calculate the norm of the attention difference value;

[0138] Among them, the norm is used to measure the distance between adversarial cross-modal attention and noisy cross-modal attention.

[0139] In some embodiments, the adversarial cross-modal attention corresponding to the adversarial image and the adversarial cross-modal attention corresponding to the noisy image are represented as an attention matrix, and the norm is used to measure the spatial distance between the adversarial cross-modal attention and the noisy cross-modal attention.

[0140] In some embodiments, the norm is The norm (Euclidean norm), which is the square root of the sum of the squares of the vector (attention matrix), is used to measure the size of the difference between the adversarial cross-modal attention and the noisy cross-modal attention. The norm value corresponding to the norm provides a quantitative measure. Optionally, a smaller norm value indicates that the attention distributions of the adversarial cross-modal attention and the noisy cross-modal attention are similar, while a larger norm value indicates that the attention distributions of the adversarial cross-modal attention and the noisy cross-modal attention are significantly different.

[0141] In some embodiments, calculating the norm of the attention difference value can also be understood as calculating the loss function of the attention difference value (the two-norm distance between the attention difference values). Optionally, the norm of the attention difference value can be expressed as:

[0142]

[0143] in, represents the adversarial image, represents a noisy image, Represents text, represents the visual language model, represents the adversarial cross-modal attention corresponding to the adversarial image, represents the noisy cross-modal attention corresponding to the noisy image, represents the attention difference value, Represents the norm of the attention difference value.

[0144] Step 241b: Calculate the norm of the attention difference value for each pixel in the adversarial image;

[0145] In some embodiments, the norm of the attention difference value can be considered as the loss function of the attention difference value. The norm of the attention difference value is the gradient of each pixel in the adversarial image, which can be understood as the gradient of the loss function for each pixel in the adversarial image. Optionally, the gradient is used to indicate the direction in which the loss function decreases fastest, that is, the gradient is used to indicate how to add the second noise to minimize the attention difference value.

[0146] Alternatively, the gradient can be expressed as:

[0147]

[0148] in, represents the gradient, represents the adversarial cross-modal attention corresponding to the adversarial image, It is represented as the noisy cross-modal attention corresponding to the noisy image, represents the gradient with respect to a pixel in the adversarial image.

[0149] Step 241c: Determine the noise adding method for each pixel based on the gradient of each pixel, add second noise to each pixel according to the noise adding method for each pixel, and obtain a purified image.

[0150] In some embodiments, the gradient direction of the gradient is determined based on a sign function, and the gradient direction is used to indicate the direction of adding a second noise to the adversarial image, that is, the second noise is added to the pixels in the adversarial image, and the pixels in the adversarial image are adjusted toward the gradient direction to reduce the attention difference value.

[0151] In some embodiments, the noise adding direction of each pixel is determined based on the gradient of each pixel in the adversarial image, which can also be understood as determining the noise adding method of each pixel based on the gradient of each pixel in the adversarial image. The noise adding direction is the same as the gradient direction, that is, the noise adding method is to add a second noise along the gradient direction on the pixel of the adversarial image. Optionally, the noise adding method includes increasing the pixel value on the pixel of the adversarial image; or, decreasing the pixel value on the pixel of the adversarial image; or, keeping the pixel value unchanged on the pixel of the adversarial image (the situation where the pixel value remains unchanged almost does not exist, and basically the pixel value of each pixel of the adversarial image will change).

[0152] Optionally, the noise adding direction (or the noise adding method for each pixel) of adding the second noise to the adversarial image is determined based on the sign function, which can be expressed as:

[0153]

[0154] in, represents the adversarial cross-modal attention corresponding to the adversarial image, It is represented as the noisy cross-modal attention corresponding to the noisy image, represents the symbolic function, Represents the gradient.

[0155] In some embodiments, the gradient is used as the input of the sign function. When the gradient is positive, that is, when the gradient of the ith pixel in the adversarial image is positive, the output of the sign function is 1, indicating that the noise addition method of the ith pixel is to increase the pixel value, and optionally, to increase the second noise corresponding to the ith pixel for the pixel value of the ith pixel. When the gradient is negative, that is, when the gradient of the ith pixel is negative, the output of the sign function is -1, indicating that the noise addition method of the ith pixel is to reduce the pixel value, and optionally, to reduce the second noise corresponding to the ith pixel for the pixel value of the ith pixel. When the gradient is zero, that is, when the gradient of the ith pixel in the adversarial image is zero, the output of the sign function is 0, indicating that the noise addition method of the ith pixel is to keep the pixel value unchanged, and optionally, to keep the pixel value of the ith pixel unchanged (this situation almost does not exist). The purified image is obtained by the above-mentioned operation of increasing the pixel value or reducing the pixel value for each pixel in the adversarial image.

[0156] In an embodiment of the present application, a sign function is used to determine the gradient direction of the gradient, and the gradient direction is used to indicate the noise adding direction of adding the second noise to the adversarial image. That is, the noise adding method of each pixel is determined based on the gradient, and the gradient is used to guide the adjustment of the pixel value, so that the adversarial image can be accurately purified. The use of a sign function can simplify the processing of the gradient, and the gradient can be directly converted into the direction of the pixel value adjustment (increase or decrease), thereby simplifying the calculation process.

[0157] In summary, the method provided in the embodiment of the present application can quantitatively evaluate the difference between adversarial cross-modal attention and noisy cross-modal attention by calculating the norm of the attention difference value. The noise addition method is determined by the gradient of each pixel, so that the purification process is targeted, and the noise addition method is determined based on the gradient of each pixel, which can be adaptively adjusted. The purified image obtained by adaptively adjusting each pixel in the adversarial image can significantly reduce the impact of adversarial disturbances, enhance the robustness and security of the visual language model, and improve its defense capabilities against adversarial attacks.

[0158] Controls the amount of noise added to the adversarial image at a pixel granularity.

[0159] In some embodiments, by calculating the gradient of each pixel in the adversarial image, the gradient includes direction information, and the gradient also includes intensity information, and the intensity information is used to indicate the magnitude of the pixel adjustment in the adversarial image.

[0160] Determine the second noise corresponding to each pixel in the adversarial image based on the intensity information of the gradient, where the intensity information is used to indicate the gradient size of each pixel in the adversarial image.

[0161] In some embodiments, the adversarial image in the adversarial sample is divided at a pixel granularity, and a second noise with different perturbation degrees (different noise sizes) is added to different pixel positions of the adversarial image based on the gradient direction. The intensity information of the gradient is used to indicate the gradient size of each pixel in the adversarial image. Optionally, pixels with larger gradients in the adversarial image should be subjected to greater perturbations, i.e., relatively larger noise is added; pixels with smaller gradients in the adversarial image should be subjected to smaller perturbations, i.e., relatively smaller noise is added.

[0162] In an embodiment of the present application, the size of the second noise added to each pixel in the adversarial image is determined by the intensity information of the gradient. The intensity information reflects the size of the gradient and can better guide the addition of the second noise, making the noise adding process more accurate.

[0163] In some embodiments, a first gradient value and a second gradient value are obtained, the first gradient value is the maximum gradient value of all pixels in the adversarial image, and the second gradient value is the minimum gradient value of all pixels in the adversarial image. Optionally, a gradient standard value is obtained based on the maximum gradient value and the minimum gradient value, and the gradient standard value is the gradient value after the gradient is normalized. Optionally, the gradient is normalized to [0, 1].

[0164] Optionally, the gradient is expressed as:

[0165]

[0166] in, represents the gradient, represents the adversarial cross-modal attention corresponding to the adversarial image, Denoted as the noisy cross-modal attention corresponding to the noisy image.

[0167] Optionally, the gradient normalized value is expressed as:

[0168]

[0169] in, represents the standard value of the gradient, represents the first gradient value, that is, the minimum gradient value, Represents the second gradient value, that is, the maximum value of the gradient.

[0170] In some embodiments, due to the large difference in gradient standard values, the gradient standard value can be represented based on the average value of the gradient standard value. Optionally, the disturbance amplitude of the second noise is determined based on the gradient standard value and the average value of the gradient standard value, and the disturbance amplitude includes a disturbance factor for determining the size of the second noise, and the size of the second noise added to each pixel in the adversarial image is determined based on the disturbance factor. Among them, the second noise size is positively correlated with the intensity information of the gradient, that is, pixels with larger gradients in the adversarial image (stronger intensity information) should be subject to greater disturbances, that is, relatively larger second noise is added; pixels with smaller gradients in the adversarial image (weaker intensity information) should have smaller disturbances, that is, relatively smaller second noise is added.

[0171] Optionally, the size of the second noise added to each pixel in the adversarial image is determined based on the disturbance factor to obtain a purified image. The purified image can be expressed as:

[0172]

[0173] in, represents the purified image, represents the adversarial image, It is used to indicate the direction of adding the second noise to the adversarial image (or the way to add noise to each pixel). represents the standard value of the gradient, represents the average value of the gradient standard value, which can be based on Determine the disturbance amplitude of the second noise, The value of is the disturbance factor, which is used to determine the second noise Optionally, 0 is the first normalized value, and 1 is the second normalized value. The perturbation factor is the maximum value of the first normalized value and the first value, the first value is the minimum value of the second normalized value and the second value, and the second value is the quotient of the gradient standard value and the average value of the gradient standard value. That is, the first value is , the second value is and Business Optionally, the disturbance factor is a value within the disturbance amplitude.

[0174] In some embodiments, the second noise is determined based on the product of a second noise value obtained by pseudo-random sampling in a second distribution interval and a disturbance factor, and the second distribution interval is a uniformly distributed interval range.

[0175] In an embodiment of the present application, by obtaining the maximum and minimum gradients of all pixels in the adversarial image and normalizing the gradient values, a standard gradient value is obtained so that gradient values ​​of different scales can be processed uniformly. The disturbance amplitude of the second noise is determined according to the standard gradient value and its average value, making the noise addition process more flexible and adaptive. This adaptive adjustment method can reasonably determine the size of the added second noise according to the actual gradient situation, that is, the larger the gradient, the larger the second noise added. This method of adding the second noise to the adversarial image improves the purification effect.

[0176] The training process of the visual language model

[0177] In some embodiments, a visual language model may be pre-trained, and the cleaned samples may be input into the visual language model to obtain a question-answering score for the visual language model, where the question-answering score is used to indicate the correct probability that the cleaned samples belong to the correct prediction result (correct category). Optionally, the question-answering score is positively correlated with the cleaned cross-modal attention of the cleaned samples.

[0178] In one possible implementation, the visual language model is trained as follows:

[0179] In some embodiments, a purified sample data set is obtained, and the purified sample data set is a collection of samples for training a visual language model. The purified sample data set includes multiple purified samples, each of which includes a purified image and text, wherein the purified image is an image obtained by the method provided in the above embodiment. The corresponding label on each purified sample is used to indicate the category to which the purified image in the purified sample belongs. Then, the purified sample is input into the visual language model, and the visual language model encodes the purified image into image features and encodes the text into text features. Optionally, the visual language model fuses the image features and the text features based on the purified cross-modal attention of the purified image. Among them, the purified cross-modal attention is used to describe the attention distribution of the purified image and text, and the visual language model can focus on the image area related to the text in the purified image based on the purified cross-modal attention.

[0180] In some embodiments, the visual language model predicts the fused features, and the fused features are features generated by fusing image features and text features based on the attention distribution of the image and text in the purified cross-modal attention. Optionally, the visual language model predicts based on the fused features to obtain a question-answering score for the purified sample, and the question-answering score is used to indicate the correct probability that the purified sample belongs to the correct prediction result (correct category). Optionally, the question-answering score is positively correlated with the purified cross-modal attention of the purified image. Optionally, a score threshold is set in the visual language model, and when the question-answering score of the visual language model is greater than the score threshold, it indicates that the purified sample belongs to the correct prediction result; when the question-answering score of the visual language model is less than or equal to the score threshold, it indicates that the purified sample belongs to an incorrect prediction result, indicating that the input purified sample belongs to an incompletely purified result. When the prediction result of the visual language model for the purified sample is an incorrect prediction result, it indicates that the purified cross-modal attention in the purified sample needs to be further adjusted, that is, random noise needs to be continued to be added to the adversarial sample to adjust the adversarial cross-modal attention of the adversarial sample.

[0181] In some embodiments, the predicted result corresponding to the question-answering score is compared with the true result (the label corresponding to the cleaned sample) to obtain a calculated loss, which is used to indicate the error between the predicted result corresponding to the question-answering score and the true result. Optionally, a back-propagation algorithm is used to adjust the parameters of the visual language model so that the predicted result of the visual language model is closer to the label corresponding to the cleaned sample. Optionally, the visual language model is optimized by a loss function. The loss function can be any loss function, for example, the loss function can be a CE loss (Cross Entropy loss) function or a mean square error (MSE) loss function.

[0182] It should be noted that the training of the visual language model provided in this embodiment is only for illustrative purposes and does not constitute a limitation on the training method of the visual language model.

[0183] In some embodiments, the visual language model can be applied to a visual question answering scenario.

[0184] In one application scenario, the sample purification method can be applied to the field of classified question answering. The purified sample is input into the visual language model, and the visual language model performs recognition based on the purified sample to obtain the correct question answer result corresponding to the purified sample.

[0185] Step 1, obtain classification adversarial samples;

[0186] Among them, classification adversarial samples are adversarial samples generated after performing adversarial attacks on clean samples. Classification adversarial samples include classification adversarial images and problem descriptions.

[0187] In some embodiments, adversarial attacks involve making small modifications or perturbations to clean examples. The modifications or perturbations in the classification adversarial examples are not noticeable, but can cause the visual language model to make incorrect predictions and generate incorrect predictions (outputs).

[0188] For example, a clean image shows a bicycle parked at the door of a building. In the question-answering scenario, the question description for the clean image is "What material is the door of this building made of?" The visual language model correctly classifies it as "wood." By applying an adversarial attack to the clean image, such as performing an adversarial attack on the clean image based on the FGSM method, a classified adversarial image can be generated. In the question-answering scenario, the question description for the classified adversarial image is "What material is the door of this building made of?" The visual language model incorrectly classifies it as "metal."

[0189] In some embodiments, the visual language model calculates adversarial cross-modal attention corresponding to the clean image based on visual tokens and language tokens in the clean image. Optionally, the clean image is encoded into image features based on the visual encoder, and the image features are aligned by the visual language model to obtain at least one visual token, and each of the at least one visual token is used to indicate the image features of the clean image; the problem description is encoded into text features based on the language model tokenizer, and the text features are aligned by the visual language model to obtain at least one language token, and each of the at least one language token is used to indicate the text features of the problem description; the clean cross-modal attention when generating a clean prediction result is calculated by the visual language model, and the clean prediction result is the prediction result of the visual language model after the fusion of at least one visual token and at least one language token.

[0190] For example, when the visual language model predicts a clean image, the generated clean prediction result is "wood". The visual language model calculates the attention weights of all visual tokens in the clean image. The visual language model divides the clean image into several regions. The image features of each region (or the image features corresponding to multiple regions) can be represented as a visual token. The attention weight represents the importance of each visual token to the answer "wood", that is, the visual language model evaluates the extent to which each visual token affects the generation of "wood".

[0191] In some embodiments, when the visual language model predicts the classified adversarial image, the adversarial cross-modal attention of the classified adversarial image is calculated. The adversarial cross-modal attention is the cross-modal attention corresponding to the classified adversarial image, which is used to describe the attention distribution of the classified adversarial image and the question description. The adversarial cross-modal attention for the classified adversarial image is similar to the clean cross-modal attention process for the clean image described above.

[0192] Step 2, adding a first noise to the classified adversarial image to obtain a noisy image;

[0193] Among them, the noisy image is an image after the first disturbance is performed on the classification adversarial image. The first noise refers to the random noise added to the classification adversarial image. The first noise is generated randomly or according to a certain rule, and the purpose is to disturb the pixel value of the classification adversarial image. In one possible implementation, the addition of the first noise is implemented by a computer program. In one possible implementation, the addition of the first noise is implemented by a noise model, for example, using a generative adversarial network, a variational autoencoder or other deep learning model to generate the first noise, and then the first noise is added to the classification adversarial image.

[0194] Step 3, calculate the noisy cross-modal attention corresponding to the noisy image;

[0195] Among them, the noisy cross-modal attention is used to describe the attention distribution of noisy images and question descriptions.

[0196] In one possible implementation, a noisy image is obtained by adding a first noise to the classified adversarial image. Compared with the adversarial cross-modal attention of the classified adversarial image, the noisy cross-modal attention of the noisy image is closer to the clean cross-modal attention of the clean image. Therefore, taking the noisy cross-modal attention of the noisy image as a reference estimation direction and guiding the adversarial cross-modal attention of the classified adversarial image to be optimized can be considered as a purification operation for the classified adversarial image to make it closer to the clean image.

[0197] For example, if the question description for a noisy image is "What material is the door of this building made of?", the visual language model may mistakenly classify it as "brick". However, the noisy cross-modal attention corresponding to the noisy image is closer to the clean cross-modal attention of the clean image.

[0198] In some embodiments, the noisy cross-modal attention corresponding to the noisy image is calculated. Noisy cross-modal attention refers to how the visual language model pays attention to the attention size (or attention weight) of the noisy image in the noisy sample when generating a prediction result based on the noisy sample. The adversarial cross-modal attention for the noisy image is similar to the process of the noisy cross-modal attention for the noisy image described above.

[0199] Step 4: Add a second noise to the classified adversarial image based on the noisy cross-modal attention to obtain a purified image.

[0200] The purified image is an image after the second perturbation is performed on the adversarial image. The second noise refers to the random noise added to the classified adversarial image. The second noise is generated randomly or according to a certain rule, and its purpose is to perturb the pixel value of the classified adversarial image.

[0201] For example, the question description for the noisy image is "What material is the door of this building made of?", and the visual language model may correctly classify it as "wood". Although the purified image may be visually noisier, the purified cross-modal attention of the purified image may be closer to the clean cross-modal attention of the clean image, so the prediction result of the visual language model for the purified image is closer to the prediction result for the clean image.

[0202] In some embodiments, by continuing to add random noise to the classified adversarial image, the adversarial cross-modal attention can be guided to approach the noisy cross-modal attention, thereby generating a purified image. Optionally, a second noise is added to the classified adversarial image with the goal of minimizing the attention difference value to obtain a purified image; the attention difference value is the difference between the noisy cross-modal attention and the adversarial cross-modal attention.

[0203] In some embodiments, the direction of adding the second noise can be determined. First, the norm of the attention difference value is calculated, and the norm of the attention difference value is used to measure the distance between the adversarial cross-modal attention and the noisy cross-modal attention. Secondly, the gradient of the attention difference value for the pixels in the classified adversarial image is calculated. Then, the gradient direction of the gradient is determined based on the sign function, which can also be understood as determining the noise addition method for each pixel based on the gradient of each pixel, and the gradient direction is used to indicate the direction of adding the second noise. Optionally, the gradient is used as the input of the sign function. When the gradient is positive, the pixel value in the adversarial image is increased in the gradient direction; when the gradient is negative, the pixel value in the adversarial image is reduced in the gradient direction, and the pixel value in the adversarial image is increased or decreased in the gradient direction to obtain a purified image after adding the second noise.

[0204] In some embodiments, the size of the second noise added can also be determined. Optionally, the gradient contains not only information about the gradient direction, but also information about the intensity. Optionally, pixels with larger gradients in the adversarial image should be subjected to greater disturbances, i.e., relatively larger noise is added; pixels with smaller gradients in the adversarial image should be subjected to smaller disturbances, i.e., relatively smaller noise is added.

[0205] In summary, the visual language model will produce incorrect predictions based on classified adversarial samples. By performing a purification operation on the classified adversarial image in the adversarial sample, a purified image is obtained, and then a purified sample can be obtained. Specifically, a noisy image is obtained by performing random noise processing on the classified adversarial image. Since the noisy cross-modal attention of the noisy image is very close to the cross-modal attention corresponding to the clean image, the noisy cross-modal attention of the noisy image can be used as the optimization target, and random noise is added to the classified adversarial image, so that the adversarial cross-modal attention of the classified adversarial image is close to the noisy cross-modal attention of the noisy image, so as to complete the purification of the classified adversarial image and obtain a purified image. The present application is guided by the noisy cross-modal attention. By adding noise to the classified adversarial image to achieve the purification of the classified adversarial image, the visual language model can obtain correct prediction results based on the purified image. This purification method enhances the robustness of the visual language model.

[0206] Quantitative results of this program

[0207] This scheme can be called FFF. Specifically, for the clean image in the clean sample , the prediction results of the visual language model It is also correct to extract clean cross-modal attention for clean images For adversarial images in adversarial samples , the prediction results of the visual language model is wrong, extracting adversarial cross-modal attention for adversarial images By comparison, we can find that the adversarial cross-modal attention of the adversarial image is significantly different from the clean cross-modal attention of the clean image, which can show that there is a significant pattern difference between the adversarial cross-modal attention graph and the clean cross-modal attention graph. By conducting feasibility verification, that is, if the clean cross-modal attention of the clean image is used as the target to optimize the adversarial image, the purification of the adversarial image can be achieved.

[0208] However, in the scenario of adversarial purification, the clean image cannot be known, and the clean cross-modal attention of the clean image cannot be extracted. Therefore, it is necessary to reasonably estimate the clean cross-modal attention of the clean image. Tests have found that by adding an infinite norm restriction to the adversarial image of the adversarial sample, The random noise is , although the prediction results of the visual language model Still wrong, but noisy cross-modal attention There is a tendency to move closer to the clean cross-modal attention of clean images. Optimizing the adversarial cross-modal attention of adversarial images may achieve adversarial image purification. Experiments have shown that this is indeed the case. By taking the adversarial cross-modal attention of randomly noisy adversarial images as the target, adding noise to the adversarial image, and obtaining a purified image, the answer is Correct and clean cross-modal attention for images It does look more like clean cross-modal attention for clean images.

[0209] Below we present some quantitative results.

[0210] First, this scheme verifies whether the clean cross-modal attention of clean images can purify adversarial images. This scheme takes the clean cross-modal attention of clean images as the target and adds noise to the adversarial images. The results are shown in Table 1. The higher the noise intensity, the better the purification effect, and the cross-modal attention corresponding to the adversarial image after adding noise is indeed closer to the clean cross-modal attention of the clean image.

[0211] Table 1

[0212]

[0213] Among them, the mean square error is the Norm. The VQA score is the visual question answering score of the visual language model based on the input image. The first row in Table 1 is the score of the visual language model (visual question answering score), and the second and third rows are the difference measures between the cross-modal attention corresponding to the adversarial image after adding noise and the clean cross-modal attention corresponding to the clean image.

[0214] Secondly, this scheme verifies the impact of the noisy image obtained by randomly adding noise to the adversarial image on the prediction results and cross-modal attention. As can be seen from Table 2, with the increase of noise intensity, the visual question answering score of the visual language model has a certain improvement but not much, but the noisy cross-modal attention corresponding to the noisy image is closer to the clean cross-modal attention of the clean image.

[0215] Table 2

[0216]

[0217] This scheme uses the noisy cross-modal attention of the randomly noisy noisy image as an estimate of the clean cross-modal attention of the clean image, and adds noise to the adversarial image. The size of the noise is randomly selected, and the estimated noisy cross-modal attention controller direction is used. The results are shown in Table 3. The first row is the random direction, and the second row and the method of this scheme are as follows. It can be seen that this scheme can indeed achieve a certain degree of performance purification. Each row estimates the noise intensity added by the clean cross-modal attention of the clean image, and each column is the noise intensity added to the purified adversarial image.

[0218] Table 3

[0219]

[0220] Previously, this scheme only controlled the direction of the noise, and the noise size was still randomly selected. This scheme further controls the noise size, and the results are shown in Table 4. The first row is the random direction, and the second row and the method of this scheme are as follows. It can be seen that compared with the optimization that only controls the direction, the effect of controlling the noise size is better.

[0221] Table 4

[0222]

[0223] Figure 7 The structure block diagram of a sample purification device provided by an embodiment of the present application is shown. The sample purification device has the function of implementing the above-mentioned sample purification method example, and the function can be implemented by hardware or by hardware executing corresponding software. The purification device can be the server introduced above, or it can be set in the server. Figure 7 As shown, the purification device may include: an acquisition module 710 , an adding module 720 and a calculation module 730 .

[0224] An acquisition module 710 is used to acquire an adversarial sample, where the adversarial sample is a sample generated after performing an adversarial attack on a clean sample, and the adversarial sample includes an adversarial image and text;

[0225] An adding module 720, configured to add a first noise to the adversarial image to obtain a noisy image;

[0226] A calculation module 730, configured to calculate a noisy cross-modal attention corresponding to the noisy image, wherein the noisy cross-modal attention is used to describe the attention distribution of the noisy image and the text;

[0227] The adding module 720 is used to add a second noise to the adversarial image based on the noisy cross-modal attention to obtain a purified image.

[0228] The calculation module 730 is further used to calculate the adversarial cross-modal attention corresponding to the adversarial image, where the adversarial cross-modal attention is used to describe the attention distribution of the adversarial image and the text;

[0229] In some embodiments, the add module 720 includes an add sub-module.

[0230] In an optional example, a submodule is added, for adding a second noise to the adversarial image with the goal of minimizing the attention difference value to obtain the purified image;

[0231] The attention difference value is the difference between the noisy cross-modal attention and the adversarial cross-modal attention.

[0232] In some embodiments, the adding submodule includes a computing unit and an adding unit.

[0233] In an optional example, a computing unit is used to calculate a norm of the attention difference value, where the norm is used to measure the distance between the adversarial cross-modal attention and the noisy cross-modal attention;

[0234] A calculation unit, configured to calculate a gradient of a norm of the attention difference value for each pixel in the adversarial image;

[0235] An adding unit is used to determine a noise adding mode for each pixel based on the gradient of each pixel, and add second noise to each pixel according to the noise adding mode for each pixel to obtain the purified image.

[0236] In an optional example, an adding unit is used to determine that the noise adding method of the i-th pixel is to increase the pixel value when the gradient of the i-th pixel in the adversarial image is positive, and to add a second noise corresponding to the i-th pixel to the pixel value of the i-th pixel, where i is a positive integer;

[0237] an adding unit, configured to determine, when the gradient of the i-th pixel is a negative value, that the noise adding method of the i-th pixel is to reduce the pixel value, and reduce the second noise corresponding to the i-th pixel from the pixel value of the i-th pixel;

[0238] An adding unit is used to obtain the purified image based on an operation of increasing a pixel value or an operation of decreasing a pixel value for each pixel in the adversarial image.

[0239] In some embodiments, the purification device further includes a determination module.

[0240] In an optional example, the gradient includes intensity information, and a determination module is used to determine the second noise corresponding to each pixel in the adversarial image based on the intensity information of the gradient, and the intensity information is used to indicate the gradient size of each pixel in the adversarial image.

[0241] In some embodiments, the determination module includes an acquisition submodule, a generation submodule, and a determination submodule.

[0242] In an optional example, an acquisition submodule is used to acquire a first gradient value and a second gradient value, wherein the first gradient value is a maximum gradient value of all pixels in the adversarial image, and the second gradient value is a minimum gradient value of all pixels in the adversarial image;

[0243] A generating submodule, configured to obtain a gradient standard value based on the first gradient value and the second gradient value, wherein the gradient standard value is a gradient value obtained by performing a normalization process on the gradient;

[0244] a determination submodule, configured to determine a disturbance amplitude of the second noise according to the gradient standard value and an average value of the gradient standard value, wherein the disturbance amplitude includes a disturbance factor for determining a magnitude of the second noise;

[0245] A determination submodule is used to determine a second noise size added to each pixel in the adversarial image based on the disturbance factor, wherein the second noise size is positively correlated with the intensity information of the gradient.

[0246] In an optional example, the disturbance factor is the maximum value of the first normalized value and the first value, the first value is the minimum value of the second normalized value and the second value, the second value is the quotient of the gradient standard value and the average value of the gradient standard value, the first normalized value is 0, and the second normalized value is 1.

[0247] In an optional example, the acquisition submodule is used to acquire a cleansed sample, wherein the cleansed sample includes the cleansed image and the text;

[0248] The generation submodule is used to generate a prediction result for the purified image based on the purified sample through a visual language model, wherein the prediction result of the purified image is a correct prediction result.

[0249] In some embodiments, the purification device includes an encoding module.

[0250] In an optional example, an encoding module is used to encode the noisy image into image features based on the visual encoder, align the image features through the visual language model to obtain at least one visual token, and each visual token in the at least one visual token is used to indicate the image feature of the noisy image;

[0251] an encoding module, configured to encode the text into text features based on the language model segmenter, align the text features through the visual language model to obtain at least one language token, wherein each language token in the at least one language token is used to indicate the text features of the text;

[0252] A calculation module is used to calculate the noisy cross-modal attention when generating a noisy prediction result through the visual language model, and the noisy prediction result is the prediction result of the visual language model after the multiple visual tokens and the multiple language tokens are fused.

[0253] In an optional example, the first noise is determined based on a first noise value obtained by pseudo-random sampling in a first distribution interval, and the first distribution interval is a uniformly distributed interval range.

[0254] In an optional example, the second noise is determined based on a second noise value obtained by pseudo-random sampling in a second distribution interval, and the second distribution interval is a uniformly distributed interval range.

[0255] It should be noted that the specific definition of the one or more sample purification devices provided above can refer to the definition of the sample purification method above, which will not be repeated here. Each module of the above purification device can be implemented in whole or in part by software, hardware and a combination thereof. Each module can be embedded in the processor of the computer device in the form of hardware or independent of it, or it can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the corresponding operations of each module.

[0256] The serial numbers of the above-mentioned embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.

[0257] Figure 8 The block diagram of the structure of a computer device 1200 shown in an exemplary embodiment of the present application is shown. The computer device can be used to implement the sample purification method provided in the above embodiment. The computer device 1200 includes a central processing unit (CPU) 1201, a system memory 1204 including a random access memory (RAM) 1202 and a read-only memory (ROM) 1203, and a system bus 1205 connecting the system memory 1204 and the central processing unit 1201. The computer device 1200 also includes a basic input / output system (I / O system) 1206 that helps transmit information between various devices in the computer device, and a large-capacity storage device 1207 for storing an operating system 1213, an application program 1214 and other program modules 1215.

[0258] The basic input / output system 1206 includes a display 1208 for displaying information and an input device 1209 such as a mouse and a keyboard for user inputting information. The display 1208 and the input device 1209 are connected to the central processing unit 1201 through an input / output controller 1210 connected to the system bus 1205. The basic input / output system 1206 may also include an input / output controller 1210 for receiving and processing inputs from a plurality of other devices such as a keyboard, a mouse, or an electronic stylus. Similarly, the input / output controller 1210 also provides output to a display screen, a printer, or other types of output devices.

[0259] The mass storage device 1207 is connected to the central processing unit 1201 via a mass storage controller (not shown) connected to the system bus 1205. The mass storage device 1207 and its associated computer-readable storage medium provide non-volatile storage for the terminal device 1200. That is, the mass storage device 1207 may include a computer-readable storage medium (not shown) such as a hard disk or a compact disc read-only memory (CD-ROM) drive.

[0260] Without loss of generality, the computer-readable storage medium may include computer storage media and communication media. Computer storage media include volatile and non-volatile, removable and non-removable media implemented by any method or technology for storing information such as computer-readable storage instructions, data structures, program modules or other data. Computer storage media include RAM, ROM, Erasable Programmable Read Only Memory (EPROM), Electronically Erasable Programmable Read-Only Memory (EEPROM), flash memory or other solid-state storage technology, CD-ROM, Digital Versatile Disc (DVD) or other optical storage, cassettes, tapes, disk storage or other magnetic storage devices. Of course, those skilled in the art will know that the computer storage medium is not limited to the above. The above-mentioned system memory 1204 and mass storage device 1207 can be collectively referred to as memory.

[0261] The memory stores one or more programs, and the one or more programs are configured to be executed by one or more central processing units 1201. The one or more programs contain instructions for implementing the above-mentioned method embodiments. The central processing unit 1201 executes the one or more programs to implement the sample purification method provided by the above-mentioned each method embodiment.

[0262] According to various embodiments of the present application, the computer device 1200 can also be connected to a remote terminal device on a network through a network such as the Internet. That is, the computer device 1200 can be connected to the network 1212 through the network interface unit 1211 connected to the system bus 1205, or the network interface unit 1211 can be used to connect to other types of networks or remote terminal device systems (not shown).

[0263] The memory also includes one or more programs, which are stored in the memory and include steps for performing sample purification performed by the terminal device in the method provided in the embodiment of the present application.

[0264] An embodiment of the present application also provides a computer-readable storage medium, in which at least one computer program is stored. The at least one computer program is loaded and executed by a processor to implement the sample purification method provided by the above-mentioned method embodiments.

[0265] An embodiment of the present application also provides a computer program product, which includes a computer program, and the computer program is stored in a computer-readable storage medium; the computer program is read and executed from the computer-readable storage medium by a processor of a computer device, so that the computer device executes to implement the sample purification method provided by the above-mentioned method embodiments.

[0266] It will be understood that in the specific implementation of the present application, the data involved, historical data, and data related to user data processing such as portraits related to user identity or characteristics, when the above embodiments of the present application are applied to specific products or technologies, need to obtain user permission or consent, and the collection, use and processing of relevant data need to comply with relevant laws, regulations and standards of relevant countries and regions.

[0267] It should be noted that, unless otherwise clearly defined herein, all terms used in the claims are interpreted according to their ordinary meanings in the technical field. Unless otherwise clearly stated, all references to "an element, device, component, equipment, step, etc." will be openly interpreted as referring to at least one instance of an element, device, component, equipment, step, etc. Unless explicitly stated, the steps of any method disclosed herein do not have to be performed in the exact order disclosed.

[0268] It should be understood that the "plurality" mentioned in this article refers to two or more. "And / or" describes the association relationship of the associated objects, indicating that there can be three relationships. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. The character " / " generally indicates that the associated objects are in an "or" relationship.

Claims

1. A sample purification method, characterized in that: The method comprises: Acquire an adversarial sample, where the adversarial sample is a sample generated after performing an adversarial attack on a clean sample, and the adversarial sample includes an adversarial image and text; Adding a first noise to the adversarial image to obtain a noisy image; Calculating a noisy cross-modal attention corresponding to the noisy image, where the noisy cross-modal attention is used to describe the attention distribution of the noisy image and the text; A second noise is added to the adversarial image based on the noisy cross-modal attention to obtain a purified image.

2. The method according to claim 1, characterized in that The method further comprises: Calculating an adversarial cross-modal attention corresponding to the adversarial image, where the adversarial cross-modal attention is used to describe the attention distribution of the adversarial image and the text; The adding a second noise to the adversarial image based on the noise-added cross-modal attention to obtain a purified image includes: With the goal of minimizing the attention difference value, adding a second noise to the adversarial image to obtain the purified image; The attention difference value is the difference between the noisy cross-modal attention and the adversarial cross-modal attention.

3. The method according to claim 2, characterized in that The step of adding a second noise to the adversarial image with the goal of minimizing the attention difference value to obtain the purified image includes: Calculating a norm of the attention difference value, where the norm is used to measure the distance between the adversarial cross-modal attention and the noisy cross-modal attention; Calculate the gradient of the norm of the attention difference value for each pixel in the adversarial image; A noise adding method for each pixel is determined based on the gradient of each pixel, and a second noise is added to each pixel according to the noise adding method for each pixel to obtain the purified image.

4. The method according to claim 3, characterized in that: The step of determining a noise adding method for each pixel based on the gradient of each pixel, and adding a second noise to each pixel according to the noise adding method for each pixel to obtain the purified image includes: When the gradient of the i-th pixel in the adversarial image is a positive value, determining that the noise adding method of the i-th pixel is to increase the pixel value, and adding the second noise corresponding to the i-th pixel to the pixel value of the i-th pixel, where i is a positive integer; When the gradient of the i-th pixel is a negative value, determining that the noise adding method of the i-th pixel is to reduce the pixel value, and reducing the second noise corresponding to the i-th pixel from the pixel value of the i-th pixel; The cleaned image is obtained based on an operation of increasing a pixel value or a operation of decreasing a pixel value for each pixel in the adversarial image.

5. The method according to claim 3 or 4, characterized in that: The gradient includes intensity information, and the method further includes: The second noise corresponding to each pixel in the adversarial image is determined based on intensity information of the gradient, where the intensity information is used to indicate the gradient size of each pixel in the adversarial image.

6. The method according to claim 5, characterized in that The determining, based on the intensity information of the gradient, the second noise corresponding to each pixel in the adversarial image comprises: Obtaining a first gradient value and a second gradient value, wherein the first gradient value is a maximum gradient value of all pixels in the adversarial image, and the second gradient value is a minimum gradient value of all pixels in the adversarial image; obtaining a gradient standard value based on the first gradient value and the second gradient value, wherein the gradient standard value is a gradient value obtained by normalizing the gradient; Determining a disturbance amplitude of the second noise according to the gradient standard value and an average value of the gradient standard value, wherein the disturbance amplitude includes a disturbance factor for determining a magnitude of the second noise; A second noise size added to each pixel in the adversarial image is determined based on the disturbance factor, and the second noise size is positively correlated with the intensity information of the gradient.

7. The method according to claim 6, characterized in that The disturbance factor is the maximum value between the first normalized value and the first value, the first value is the minimum value between the second normalized value and the second value, the second value is the quotient of the gradient standard value and the average value of the gradient standard value, the first normalized value is 0, and the second normalized value is 1.

8. The method according to any one of claims 1 to 7, characterized in that: The method further comprises: Acquire a cleansed sample, wherein the cleansed sample includes the cleansed image and the text; A prediction result for the purified image is generated based on the purified sample by using a visual language model, and the prediction result for the purified image is a correct prediction result.

9. The method according to any one of claims 1 to 7, characterized in that: The visual language model includes a visual encoder and a language model segmenter; The calculating the noisy cross-modal attention corresponding to the noisy image includes: Encoding the noisy image into image features based on the visual encoder, aligning the image features through the visual language model to obtain at least one visual token, each of the at least one visual token is used to indicate the image feature of the noisy image; Encoding the text into text features based on the language model segmenter, aligning the text features through the visual language model to obtain at least one language token, each of the at least one language token is used to indicate the text features of the text; The noisy cross-modal attention when generating a noisy prediction result is calculated by the visual language model, and the noisy prediction result is a prediction result after the visual language model is fused with the at least one visual token and the at least one language token.

10. The method according to any one of claims 1 to 9, characterized in that: The first noise is determined based on a first noise value obtained by pseudo-random sampling in a first distribution interval, and the first distribution interval is a uniformly distributed interval range.

11. The method according to any one of claims 1 to 9, characterized in that: The second noise is determined based on a second noise value obtained by pseudo-random sampling in a second distribution interval, and the second distribution interval is a uniformly distributed interval range.

12. A sample purification device, characterized in that: The device comprises: An acquisition module, used to acquire adversarial samples, wherein the adversarial samples are samples generated after performing adversarial attacks on clean samples, and the adversarial samples include adversarial images and texts; An adding module, configured to add a first noise to the adversarial image to obtain a noisy image; A calculation module, used to calculate the noisy cross-modal attention corresponding to the noisy image, where the noisy cross-modal attention is used to describe the attention distribution of the noisy image and the text; A control module is used to add a second noise to the adversarial image based on the noisy cross-modal attention to obtain a purified image.

13. A computer device, characterized in that: The computer device comprises: a processor and a memory, wherein at least one computer program is stored in the memory, and the at least one computer program is loaded and executed by the processor to implement the sample purification method according to any one of claims 1 to 11.

14. A computer storage medium, characterized in that: At least one computer program is stored in the computer storage medium, and the at least one computer program is loaded and executed by the processor to implement the sample purification method according to any one of claims 1 to 11.

15. A computer program product, characterized in that The computer program product comprises a computer program, which is stored in a computer-readable storage medium; the computer program is read and executed from the computer-readable storage medium by a processor of a computer device, so that the computer device executes the sample purification method according to any one of claims 1 to 11.

Images