Black box attack confrontation sample generation method and device

Abstract

The invention discloses a black box attack confrontation sample generation method and device, and relates to the technical field of artificial intelligence security and confrontation sample generation. The problems that under the black box attack, an adversarial sample generation method depends too much on a model, redundant disturbance and unnecessary pixel tampering are likely to be caused, the query frequency is high, and the attack success rate is low are solved. The method comprises the steps of obtaining a first adversarial sample based on a first key semantic pixel, and inputting the first adversarial sample into a first visual large model to obtain a current decision tag; obtaining a first loss function according to the current decision label and the accurate category label; determining a first gradient of the first adversarial sample according to the first loss function, obtaining first adversarial disturbance according to the first gradient, the first key semantic pixel and the update step length, and obtaining a final adversarial sample according to the first adversarial disturbance and the first adversarial sample, the final decision label obtained by inputting the final adversarial sample into the first visual large model is different from the accurate category label.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence security and adversarial sample generation technology, and more specifically to a method and device for generating black-box attack adversarial samples. Background Art

[0002] With the development of artificial intelligence technology, visual big models continue to emerge with super strong semantic understanding and content generation capabilities. With wide applications in many key fields such as image understanding, autonomous driving, and medical image analysis, its security issues have become a research focus. Existing research is mainly aimed at white-box attack scenarios. The attacker has a complete understanding of the visual model, including the structure of the target model (such as the number of layers of the neural network, the number of neurons in each layer, etc.), parameters (weights and biases), and the distribution of training data. The gradient feedback of the input image to the visual big model is used to iteratively generate adversarial samples for designing specific target attacks and non-specific target attack strategies. In addition, in the black-box attack scenario, the attacker does not know the internal structure and weight parameters of the visual big model, and often randomly selects pixels for multi-value tampering to generate adversarial samples for black-box attacks. At the same time, white-box attacks are carried out on known big models to generate adversarial samples, and the attack transferability of adversarial samples is used to mislead unknown big models with the same transformer and convolution structure to achieve black-box attacks.

[0003] However, both the random tampering trial and error method and the attack transferability method lack the decision feedback information of the black box model, and the adversarial sample generation efficiency and attack success rate are low. The adversarial samples generated for a certain large visual model are difficult to effectively attack other large visual models, which seriously limits the comprehensive evaluation of the security of large visual models and the formulation of protection strategies. Therefore, there is an urgent need for a method that can generate adversarial samples with high transferability in a black box environment. Summary of the invention

[0004] An embodiment of the present invention provides a method and device for generating adversarial samples for a black-box attack, so as to solve the problem that the current adversarial sample generation method under a black-box attack relies too much on a model, which easily causes redundant disturbances and unnecessary pixel tampering, resulting in a high number of queries and a low attack success rate.

[0005] An embodiment of the present invention provides a method for generating a black-box attack adversarial sample, comprising:

[0006] The original image is input into a set first visual large model to obtain an accurate category label, and the initial adversarial sample is input into an unknown second visual large model to obtain an initial classification value; the initial adversarial sample obtains the first key semantic pixel of the initial adversarial sample based on the second visual large model and the initial classification value by masking pixels one by one; wherein the adversarial sample is obtained according to the original image;

[0007] Obtain a first adversarial sample based on the first key semantic pixel, input the first adversarial sample into the first visual large model to obtain a current decision label; obtain a first loss function according to the current decision label and the accurate category label;

[0008] A first gradient of the first adversarial sample is determined according to the first loss function, a first adversarial perturbation is obtained according to the first gradient, the first key semantic pixel and the update step size, and a final adversarial sample is obtained according to the first adversarial perturbation and the first adversarial sample, wherein a final decision label obtained by inputting the final adversarial sample into the first large visual model is different from the accurate category label.

[0009] Preferably, obtaining a final adversarial sample according to the first adversarial perturbation and the first adversarial sample specifically includes:

[0010] Obtain an updated adversarial sample according to the first adversarial perturbation and the first adversarial sample, input the updated adversarial sample into the first visual macro model, and obtain an updated decision label corresponding to the updated adversarial sample;

[0011] An updated loss function is constructed according to the updated decision label and the accurate category label; an updated gradient of the updated adversarial sample is determined according to the updated loss function, an updated adversarial perturbation is obtained according to the updated gradient, the first key semantic pixel and the update step size, and the updated adversarial sample is updated at the first key semantic pixel to obtain the final adversarial sample.

[0012] Preferably, determining the first gradient of the first adversarial sample according to the first loss function specifically includes:

[0013] The first gradient of the first loss function with respect to the first adversarial sample is determined by a back propagation algorithm, and the first gradient is as follows:

[0014]

[0015] in, represents the first gradient, represents the first adversarial example, represents the initial adversarial sample, represents the mask image, represents the first loss function, represents the rate of change of the first loss function relative to the first adversarial example, represents the current decision label, Represents the exact category label.

[0016] Preferably, the first counter-perturbation is as follows:

[0017]

[0018] in, represents the first adversarial perturbation, represents the update step size, represents the first key semantic pixel, Represents the first gradient.

[0019] Preferably, obtaining a first loss function according to the current decision label and the accurate category label specifically includes:

[0020] according to The first loss function is constructed by using a distance function, and the current decision label and the accurate category label are numerically represented to obtain the first loss function as shown below:

[0021]

[0022] Alternatively, the first loss function is constructed according to the cross entropy, the current decision label and the accurate category label are numerically represented, and the probability distribution vectors of the current decision label and the accurate category label are determined respectively, to obtain the first loss function shown below:

[0023]

[0024] in, represents the first loss function, represents the current decision label, represents the accurate category label, represents the first loss function, The base is The logarithmic function of Indicates taking the largest distance, Indicates taking the minimum value.

[0025] Preferably, the initial adversarial sample obtains the first key semantic pixel of the initial adversarial sample based on the second visual large model and the initial classification value by masking pixels one by one, specifically including:

[0026] The initial adversarial sample masks pixels in a set manner, inputs the masked pixel image into the second visual large model, obtains multiple decision classification values ​​in turn, determines the difference between each decision classification value and the initial classification value, and determines the masked pixel corresponding to the maximum difference as the first key semantic pixel.

[0027] Preferably, the difference between each of the decision classification values ​​and the initial classification value is determined by the following formula:

[0028]

[0029] in, Represents the masked pixel image The decision classification value output by the second visual model, Represents the masked pixel image, Denotes the initial adversarial sample as The initial classification value output after passing through the second visual model, represents the initial adversarial sample.

[0030] Preferably, obtaining a first adversarial sample based on the first key semantic pixel specifically includes:

[0031] Creating a mask image of the same size as the initial adversarial sample, setting the position of the first key semantic pixel to 1 in the mask image, and obtaining the pixel value of each coordinate of the mask image; combining the initial adversarial sample and the mask image to obtain a first adversarial sample;

[0032] The pixel value of each coordinate of the mask image is as follows:

[0033]

[0034] in, Represents the mask image The median coordinate is The pixel value of represents the mask image, Represents the first key semantic pixel.

[0035] Preferably, the first large visual model is GPT-4Vision or BERT;

[0036] The current decision labels are as follows:

[0037]

[0038] Among them, the represents the current decision label, represents the first adversarial example, A function representing a first visual large model.

[0039] An embodiment of the present invention provides a device for generating a black-box attack adversarial sample, comprising:

[0040] A first obtaining unit is used to input the original image into a set first visual large model to obtain an accurate category label, and input the initial adversarial sample into an unknown second visual large model to obtain an initial classification value; the initial adversarial sample obtains a first key semantic pixel of the initial adversarial sample based on the second visual large model and the initial classification value by masking pixels one by one; wherein the adversarial sample is obtained according to the original image;

[0041] A second obtaining unit is used to obtain a first adversarial sample based on the first key semantic pixel, input the first adversarial sample into the first visual large model to obtain a current decision label; and obtain a first loss function according to the current decision label and the accurate category label;

[0042] A third obtaining unit is used to determine a first gradient of the first adversarial sample according to the first loss function, obtain a first adversarial perturbation according to the first gradient, the first key semantic pixel and an update step size, and obtain a final adversarial sample according to the first adversarial perturbation and the first adversarial sample, wherein a final decision label obtained by inputting the final adversarial sample into the first large visual model is different from the accurate category label.

[0043] An embodiment of the present invention provides a computer device, comprising a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor executes any one of the above-mentioned methods for generating a black-box attack adversarial sample.

[0044] An embodiment of the present invention provides a computer-readable storage medium storing a computer program. When the computer program is executed by a processor, the processor executes any one of the above-mentioned methods for generating a black-box attack adversarial sample.

[0045] The embodiment of the present invention provides a method and device for generating adversarial samples for black-box attacks, which uses a known first visual large model to construct a first loss function without a specific target attack, thereby obtaining the first gradient information of the first adversarial sample, and simultaneously uses the decision feedback information of the unknown second visual large model to obtain the first key semantic pixel of the first adversarial sample, and fuses the above-mentioned first gradient information and the first key semantic pixel to generate a final adversarial sample, which can achieve an efficient black-box attack on the second visual large model; the method verifies that the interactive feedback and information iteration between the first visual large model and the second visual large model can enhance the attack transferability of the final adversarial sample and achieve an attack with a high success rate. The method solves the problem that the current adversarial sample generation method under black-box attack relies too much on the model, which easily causes redundant disturbances and unnecessary pixel tampering, resulting in a high number of queries and a low attack success rate. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0047] Figure 1 A schematic diagram of a flow chart of a method for generating a black-box attack adversarial sample provided by an embodiment of the present invention;

[0048] Figure 2 A schematic diagram of the structure of a device for generating black-box attack adversarial samples provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0049] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0050] Figure 1 A schematic diagram of a method for generating a black box attack adversarial sample according to an embodiment of the present invention is provided below. Figure 1 The method for generating a black box attack adversarial sample provided by an embodiment of the present invention is introduced. Figure 1 As shown, the method comprises the following steps:

[0051] Step 101, inputting the original image into a set first visual large model to obtain an accurate category label, and inputting the initial adversarial sample into an unknown second visual large model to obtain an initial classification value; the initial adversarial sample obtains the first key semantic pixel of the initial adversarial sample based on the second visual large model and the initial classification value by masking pixels one by one; wherein the adversarial sample is obtained according to the original image;

[0052] Step 102, obtaining a first adversarial sample based on the first key semantic pixel, inputting the first adversarial sample into the first visual large model to obtain a current decision label; obtaining a first loss function according to the current decision label and the accurate category label;

[0053] Step 103: determine the first gradient of the first adversarial sample according to the first loss function, obtain the first adversarial perturbation according to the first gradient, the first key semantic pixel and the update step size, and obtain the final adversarial sample according to the first adversarial perturbation and the first adversarial sample, wherein the final decision label obtained by inputting the final adversarial sample into the first visual large model is different from the accurate category label.

[0054] Before step 101, the original image is first obtained. The original image comes from public image datasets such as MNIST (Modified National Institute of Standards and Technology database, MNIST dataset), COCO (Common Objects in Context, COCO dataset), Imagenet, etc. It can also come from image datasets captured by a camera or crawled from a network gallery.

[0055] Specifically, the public image dataset can be the MNIST dataset, which is mainly used for research and tasks related to digital recognition. It contains a large number of images of handwritten numbers, which is very helpful for training and testing digital classification models. The COCO dataset focuses on tasks such as object detection, segmentation, and captioning, and has rich object categories and complex scene images. ImageNet is a very large-scale image dataset used for many computer vision tasks such as image classification and target detection.

[0056] The images captured by cameras are usually closely related to the actual scene. For example, in monitoring systems, images captured by cameras can be used for applications such as personnel recognition and behavior analysis; in the field of autonomous driving, images of roads, traffic signs, and vehicles captured by on-board cameras are important data sources for environmental perception.

[0057] Web image crawling can be done by writing a web crawler to get images from Internet image libraries. The themes and uses of these images are very wide.

[0058] In the embodiment of the present invention, the original image is the basic data processed by the visual big model, and the visual big model here can be a known visual big model or an unknown visual big model. In normal computer vision tasks, if image classification is required, the visual big model will learn the features of the original image to determine the category to which the original image belongs; if target detection is required, the visual big model will identify information such as the location and category of the target object in the original image.

[0059] In practical applications, adversarial examples are generated based on the original image after specific perturbations (usually small changes that are difficult for the human eye to detect). The purpose is to make a well-trained machine learning model produce incorrect output when processing these adversarial examples. For example, in an image classification task, an original image that was correctly classified as a "cat" may be misclassified as a "dog" by the model after the adversarial example is processed.

[0060] In an embodiment of the present invention, the original image is initialized to obtain an initial adversarial sample. Since the original image itself has features that can be correctly recognized by the visual large model, fine-tuning the initial adversarial sample on this basis can better control the direction and degree of generation of the initial adversarial sample; by starting from the original image, it can ensure that the initial adversarial sample has a high degree of semantic and visual similarity with the original image; generating the final adversarial sample from the original image helps to better understand the impact of changes in each step of the final adversarial sample generation process on the output of the unknown visual large model.

[0061] In step 101, an original image obtained from a data set is input into a known first visual large model, and an accurate category label of the original image can be obtained through the first visual large model.

[0062] In an embodiment of the present invention, the first large visual model is a model based on the Transformer architecture, such as GPT-4Vision (Generative Pretrained Transformer -4 with Vision, GPT-4V) or BERT (Bidirectional Encoder Representations from Transformers, BERT model).

[0063] In the embodiment of the present invention, the accurate category label can be expressed by the following formula:

[0064]

[0065] in, represents the accurate category label, represents the original image, Represents the function of processing the original image of the first visual model.

[0066] For example, the original image It is the input data of the first vision model. The first vision model is based on the Transformer architecture model, such as GPT-4 Vision. Specifically, the original image is a scene of a sheepdog playing in the park. This picture comes from a photo. The original image is provided as input to GPT-4 Vision. After GPT-4 Vision receives the original image, it uses its internal complex neural network based on the Transformer architecture to process the original image. The model will recognize various primitives in the original image, including animal species, scenes and other features. By analyzing and learning these features, the model will output a result. ,here Represents the category label "sheepdog", that is:

[0067]

[0068] here Indicates that GPT-4 Vision classifies the input original image as an image of a sheepdog.

[0069] For example, the original image It is the input data of the first visual model. The first visual model is based on the Transformer architecture model, such as BERT. Specifically, the original image is a scene of a collie playing in the park. This kind of picture comes from a photo. The original image is provided as input to BERT. After BERT receives the original image, it first converts the image into a text description, and then inputs the text description into the fine-tuned BERT. After calculation, BERT outputs a probability distribution, such as [0.8, 0.1, 0.1], which respectively represent the probability that the image is a collie, lawn, and toy. The collie category with the highest probability is selected as the prediction result. Here, the category with the highest probability is the first category, that is, the collie, so ,Right now:

[0070]

[0071] here This means that BERT classifies the input original image as an image of a sheepdog.

[0072] It should be noted that when the original image Before inputting into BERT, it is necessary to build a BERT-based image classification model to identify images containing collies and determine whether they are collies, toys, and scenes.

[0073] Furthermore, the initial adversarial sample determined based on the original image is input into the unknown second visual large model to obtain an initial classification value. It should be noted that the unknown here means that the processor is not clear about the specific structure, parameters and other details of the second visual large model, but in this embodiment, the processor can know the initial adversarial sample input into the second visual large model and determine the output result of the second visual large model. Furthermore, the initial adversarial sample here is the same as the original image, that is:

[0074]

[0075] in, represents the original image, represents the initial adversarial sample.

[0076] In an embodiment of the present invention, in order to find out which pixels included in the initial adversarial sample are most critical to the decision of the second visual large model, the initial adversarial sample masks pixels in a set manner, and the masked pixel images are input into the second visual large model in sequence, so that multiple decision classification values ​​can be obtained.

[0077] Furthermore, according to the difference between each decision classification value and the initial classification value, the masked pixel image corresponding to the maximum difference can be found, and then the masked pixel corresponding to the masked pixel image is determined as the first key semantic pixel. In the embodiment of the present invention, the first key semantic pixel has the greatest influence on the decision of the second visual large model. By finding such a pixel, it can be understood which parts of the input image the second visual large model pays more attention to when making a decision.

[0078] For example, the initial adversarial example is , mask the initial adversarial samples in turn according to the setting method For the initial adversarial sample Any pixel point in has coordinates , where the masked pixel image is determined as . The masked pixel image The second visual model gives a decision classification value for each input masked pixel image. Accordingly, the initial adversarial sample is Input into the second visual model, the initial classification value is determined as . Further, through , calculate the difference between each decision classification value of each masked pixel image and the initial classification value of the initial adversarial sample, determine the masked pixel image corresponding to the maximum difference among all the differences, and then determine the corresponding masked pixel according to the masked pixel image ,Finally, the determined masked pixel is determined as the first key semantic pixel.

[0079] For example, suppose the initial adversarial example is This is an image of a cat. The known large visual model originally used to identify cats may misidentify the cat as a dog when facing this initial adversarial sample. The following needs to input the initial adversarial sample into an unknown large visual model to find the key semantic pixels.

[0080] Specifically, the pixels of the initial adversarial sample are masked in sequence according to the setting method, starting with the first pixel in the upper left corner, with coordinates Here, the masked pixel image is determined Then Input to the unknown second visual model, the second visual model outputs the decision classification value Then, the next pixel is masked in order from left to right and from top to bottom, and the masked pixel image is input into the second visual model to obtain multiple decision classification values. It is also input into the second visual model, and the initial classification value is determined as .

[0081] pass , calculate the difference between the decision classification value of each masked pixel image and the initial classification value of the initial adversarial sample, find the largest difference among all the differences, and assume that the pixel coordinates corresponding to this maximum difference are , then this pixel It is the first key semantic pixel determined.

[0082] In step 102, since the first key semantic pixel is determined in step 101, a first adversarial sample can be obtained based on the first key semantic pixel.

[0083] In an embodiment of the present invention, one method is to obtain the first adversarial sample by creating a mask image. Specifically, a mask image of the same size as the initial adversarial sample is created, and in the mask image, the position of the first key semantic pixel is set to 1, and the pixel value of each coordinate of the mask image is obtained; the initial adversarial sample and the mask image are combined together to obtain the first adversarial sample; the pixel value of each coordinate of the mask image is as follows:

[0084]

[0085] in, Represents the mask image The median coordinate is The pixel value of represents the mask image, Represents the first key semantic pixel.

[0086] Another way is to obtain the first adversarial sample based on weighted fusion. Specifically, a weight parameter is set , which is used to control the influence of the first key semantic pixel on the new representation. Based on the first key pixel, a weighted image with the same size as the initial adversarial sample is constructed. , in the weighted image For the position of the first key pixel, set its value to , and the values ​​of the remaining pixel positions are set to , the weighted image Multiply the initial adversarial sample element by element to get the first adversarial sample. The first adversarial sample is represented as , the pixel value of each coordinate of the weighted image is as follows:

[0087]

[0088] in, Represents a weighted image The median coordinate is The pixel value of represents the first key semantic pixel, Represents element-wise multiplication.

[0089] It should be noted that the above-mentioned methods of constructing the first adversarial sample by creating a mask image or weighted fusion are only two methods of constructing the first adversarial sample provided in the embodiments of the present invention. In the subsequent embodiments, taking the construction of the first adversarial sample by creating a mask image as an example, the method for generating the black box attack adversarial sample provided by the embodiments of the present invention is specifically introduced. In the embodiments of the present invention, the method for constructing the first adversarial sample is not specifically limited.

[0090] Furthermore, the first adversarial sample is input into the first visual large model. Since the first visual large model is known, after the first adversarial sample is input into the first visual large model, the first visual large model outputs the current decision label corresponding to the first adversarial sample.

[0091] Specifically, the first adversarial sample consists of the initial adversarial sample and the mask image Composition, denoted as , mask image is an image of the same size as the initial adversarial sample, and its mask image is used to highlight the first key semantic pixel. The input data is input to the first visual model. The first visual model is based on the Transformer architecture model, such as GPT-4 Vision.

[0092] For example, if the initial adversarial sample is an image of an apple, which comes from a public dataset; the first adversarial sample As input, GPT-4 Vision receives the first adversarial sample and processes it using its internal complex neural network based on the Transformer architecture to analyze the first adversarial sample. Features and mask images The key position corresponding to the first semantic pixel indicated is calculated and reasoned within the model, and finally outputs the current decision label. In this embodiment, GPT-4Vision can accurately identify that the nature of the first adversarial example is an apple, that is:

[0093]

[0094] here Indicates that GPT-4 Vision classifies the first adversarial example input as an image of an apple.

[0095] Specifically, the first adversarial example It is the input data of the first visual model. The first visual model is based on the Transformer architecture model, such as BERT.

[0096] For example, if the first adversarial example is an image of an apple, which comes from a photo; the first adversarial sample Provided as input to BERT, in order to adapt the first adversarial sample data to BRET, we first Feature extraction is performed to obtain a feature vector, and then the feature vector is appropriately combined with the mask image M to form a format suitable for BERT input and input to BERT. BERT processes the input data. Due to the guidance of the mask image, BERT pays more attention to the coordinates of the first key semantic pixel. After calculation and reasoning within BERT, the current decision label is finally output. In this embodiment, BERT can accurately identify that the essence of the first adversarial sample is an apple, that is:

[0097]

[0098] here This indicates that BERT classifies the first adversarial example input as an image of an apple.

[0099] Further, according to the current decision label and the accurate category label, a first loss function including the current decision label and the accurate category label is constructed.

[0100] In an embodiment of the present invention, one method can be used Distance function is used to construct the first loss function. Distance (also called Euclidean distance) is a measure of the distance between two vectors. Distance is used to measure the current decision label and accurate category labels In this embodiment of the present invention, the first loss function is as follows:

[0101]

[0102] in, represents the current decision label, represents the accurate category label, represents the first loss function, Indicates taking the largest Distance is used as the first loss function.

[0103] Another way is to use cross entropy to construct the first loss function, which converts the current decision label and accurate category labels Numerical representation is performed to determine the current decision label and accurate category labels The probability distribution vector of , in this embodiment of the present invention, the first loss function is as follows:

[0104]

[0105] in, represents the current decision label, represents the accurate category label, represents the first loss function, The base is The logarithmic function of Indicates taking the minimum value as the first loss function.

[0106] For example, when there is an image classification task, the goal is to distinguish different kinds of animals, including cats, dogs, rabbits, etc.; the original image selects a clear cat image. After the original image is input into the first visual model, the output accurate category label is , in this example The first adversarial sample obtained based on the initial adversarial sample (clear cat image) and the mask image After being input into the first visual model, the first visual model outputs the current decision label In this example, it is assumed that due to the influence of adversarial perturbations, after the first adversarial sample is input into the first visual model, the current decision label output by the first visual model is The first adversarial sample is misclassified as “dog” by the first visual model.

[0107] 1) When using Distance function to construct the first loss function:

[0108] The current decision label and the accurate category label need to be numerically represented. For example, one-hot encoding can be used. Assume that "cat" is represented as [ 1,0,0] , "dog" is represented by [ 0,1,0] , "rabbit" is represented by [ 0,0,1] . Then we can determine that the accurate category label is "cat", which is expressed as y r = [ 1,0,0] , accordingly, we can also determine that the current decision label is "dog" which is expressed as y ' = [ 0,1,0] Further, according to The distance formula can be expressed as follows:

[0109]

[0110] in, is the accurate category label The three components of is the accurate category label The three components.

[0111] Further calculation can be obtained:

[0112] .

[0113] according to Distance function, when the accurate category label is "cat" and the current decision label is "dog", the first loss function constructed is .

[0114] 2) Construct the first loss function based on cross entropy:

[0115] The current decision label and the accurate category label need to be numerically represented. For example, one-hot encoding can be used. Suppose "cat" is represented as [ 1,0,0] , "dog" is represented by [ 0,1,0] , "rabbit" is represented by [ 0,0,1] . Then we can determine that the accurate category label is "cat", which is expressed as y r = [ 1,0,0] , accordingly, we can also determine that the current decision label is "dog", expressed as y ' = [ 0,1,0] . Further, according to the cross entropy, the following expression can be obtained:

[0116]

[0117] Calculate each term separately: ; ; In actual calculation, due to Meaningless, usually a minimum value is added to the probability value To avoid this situation, it is assumed here , , is approximately negative infinity. ; .

[0118] At this time, the first loss function is constructed based on the cross entropy. When the accurate category label is "cat" and the current decision label is "dog", the first loss function constructed is .

[0119] In step 103, the first gradient of the first adversarial sample is determined according to the first loss function, which is essentially to calculate the first loss function The first adversarial example about input The first gradient here is as follows:

[0120]

[0121] In the formula, Represents the current decision label, that is, the first adversarial sample of the first visual large model input The output, represents the accurate category label, Represents the mask image, through the mask image Highlight the first key semantic pixel .

[0122] In the embodiment of the present invention, by back propagation based on the chain rule, starting from the first loss function, the parameters and inputs of each layer in the first visual model are derived in turn until the first adversarial sample is obtained. The first gradient.

[0123] Furthermore, based on the first gradient obtained by back propagation of the chain rule, the adversarial perturbation is iterated at the first key semantic pixel, that is, the first adversarial perturbation is obtained based on the first gradient, the first key semantic pixel and the update step size:

[0124]

[0125] in, represents the first adversarial perturbation, represents the update step size, represents the first key semantic pixel, Represents the first gradient. In the embodiment of the present invention, the update step size in the above formula determines the amplitude of each update, and the negative sign in the formula indicates an update in the opposite direction of the first gradient. The method for generating black-box attack adversarial samples provided in the embodiment of the present invention hopes to increase the first loss function in the iterative process, and the direction of the first gradient is the fastest way to increase the first loss function.

[0126] Further, when the first adversarial perturbation is obtained, the first adversarial sample is updated at the first key semantic pixel, and an updated adversarial sample is obtained according to the first adversarial perturbation and the first adversarial sample. In the embodiment of the present invention, after obtaining an updated adversarial sample, the above steps 102 and 103 need to be re-executed.

[0127] For example, the once-updated adversarial sample is input into the first visual large model to obtain the once-updated decision label corresponding to the once-updated adversarial sample; based on the once-updated decision label and the accurate category label, a once-updated loss function including the once-updated decision label and the accurate category label is constructed; the once-updated gradient of the once-updated adversarial sample is determined according to the once-updated loss function, and the once-updated adversarial perturbation is obtained according to the once-updated gradient, the first key semantic pixel and the update step size, and the once-updated adversarial sample is updated at the first key semantic pixel to obtain a second-updated adversarial sample. If a second-updated adversarial sample is obtained, it means that in this iterative update cycle, the once-updated decision label is the same as the accurate category label; therefore, it is necessary to re-execute the above steps 102 and 103 based on the second-updated adversarial sample until the N-updated decision labels obtained based on the N-updated adversarial sample are different from the accurate category label, then the iteration can be stopped, and the N-updated adversarial sample corresponding to the N-updated decision labels is called the final adversarial sample.

[0128] It should be noted that the N-times updated decision labels here are not the same as the accurate category labels. It can be understood that the N-times updated decision labels obtained after N iterations are different from the actual, accurate category labels. For example, in an image recognition project, a large number of animal pictures need to be classified and labeled. Suppose there is a picture of a Siamese cat, then the accurate category label of this picture is "cat". After N iterations, the N-times updated decision label given by the First Vision large model for the above Siamese cat picture is "fox", which is obviously different from the accurate category label "cat".

[0129] In an embodiment of the present invention, in order to avoid confusion of terms caused by multiple updates, when step 102 and step 103 are executed for the first time, "first" is added before the terms obtained in the above steps. Correspondingly, when step 102 and step 103 are re-executed, "one update", "two updates" or "N updates" are added before the terms obtained in the above steps according to the number of re-executions, where "one", "two" and "N times" correspond to the number of re-executions respectively; at the end of the iterative update, the N updated adversarial samples corresponding to the N update decision labels obtained when the Nth update that finally determines to stop the iteration are determined as the final adversarial samples, where N represents a positive integer greater than zero.

[0130] In summary, the embodiment of the present invention provides a method and device for generating adversarial samples for black-box attacks, which uses a known first visual large model to construct a first loss function without a specific target attack, thereby obtaining the first gradient information of the first adversarial sample, and simultaneously uses the decision feedback information of the unknown second visual large model to obtain the first key semantic pixel of the first adversarial sample, and fuses the above-mentioned first gradient information and the first key semantic pixel to generate a final adversarial sample, which can achieve an efficient black-box attack on the second visual large model; the method verifies that the interactive feedback and information iteration between the first visual large model and the second visual large model can enhance the attack transferability of the final adversarial sample and achieve an attack with a high success rate. It solves the problem that the current adversarial sample generation method under black-box attack relies too much on the model, which easily causes redundant disturbances and unnecessary pixel tampering, resulting in a high number of queries and a low attack success rate.

[0131] Based on the same inventive concept, an embodiment of the present invention provides a device for generating a black-box attack adversarial sample. Since the principle of solving the technical problem of the device is similar to a method for generating a black-box attack adversarial sample, the implementation of the device can refer to the implementation of the method, and the repeated parts will not be repeated.

[0132] like Figure 2 As shown, the device includes a first obtaining unit 201 , a second obtaining unit 202 and a third obtaining unit 203 .

[0133] The first obtaining unit 201 is used to input the original image into a set first visual large model to obtain an accurate category label, and input the initial adversarial sample into an unknown second visual large model to obtain an initial classification value; the initial adversarial sample obtains the first key semantic pixel of the initial adversarial sample based on the second visual large model and the initial classification value by masking pixels one by one; wherein the adversarial sample is obtained according to the original image;

[0134] A second obtaining unit 202 is configured to obtain a first adversarial sample based on the first key semantic pixel, input the first adversarial sample into the first visual large model to obtain a current decision label; and obtain a first loss function according to the current decision label and the accurate category label;

[0135] The third obtaining unit 202 is used to determine the first gradient of the first adversarial sample according to the first loss function, obtain the first adversarial perturbation according to the first gradient, the first key semantic pixel and the update step size, and obtain the final adversarial sample according to the first adversarial perturbation and the first adversarial sample, wherein the final decision label obtained by inputting the final adversarial sample into the first visual large model is different from the accurate category label.

[0136] It should be understood that the units included in the above-mentioned device for generating a black-box attack adversarial sample are only logical divisions based on the functions implemented by the device. In practical applications, the above-mentioned units can be superimposed or split. In addition, the functions implemented by the device for generating a black-box attack adversarial sample provided in this embodiment correspond one-to-one to the method for generating a black-box attack adversarial sample provided in the above-mentioned embodiment. The more detailed processing flow implemented by the device has been described in detail in the above-mentioned method embodiment 1, and will not be described in detail here.

[0137] Another embodiment of the present invention also provides a computer device, which includes: a processor and a memory; the memory is used to store computer program code, and the computer program code includes computer instructions; when the processor executes the computer instructions, the electronic device executes each step of the method for generating black-box attack adversarial samples in the method flow described in the above method embodiment.

[0138] Another embodiment of the present invention further provides a computer-readable storage medium, which stores computer instructions. When the computer instructions are executed on a computer device, the computer device executes each step of the method for generating black-box attack adversarial samples in the method flow described in the above method embodiment.

[0139] Although the preferred embodiments of the present invention have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present invention.

[0140] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

Claims

1. A method for generating a black-box attack adversarial sample, characterized in that: include: The original image is input into the set first visual model to obtain the accurate category label, and the initial adversarial sample is input into the unknown second visual model to obtain the initial classification value; The initial adversarial sample obtains a first key semantic pixel of the initial adversarial sample based on the second visual large model and the initial classification value by masking pixels one by one; wherein the adversarial sample is obtained according to the original image; Obtain a first adversarial sample based on the first key semantic pixel, input the first adversarial sample into the first visual large model to obtain a current decision label; obtain a first loss function according to the current decision label and the accurate category label; A first gradient of the first adversarial sample is determined according to the first loss function, a first adversarial perturbation is obtained according to the first gradient, the first key semantic pixel and the update step size, and a final adversarial sample is obtained according to the first adversarial perturbation and the first adversarial sample, wherein a final decision label obtained by inputting the final adversarial sample into the first large visual model is different from the accurate category label.

2. The method according to claim 1, characterized in that The obtaining a final adversarial sample according to the first adversarial perturbation and the first adversarial sample specifically includes: Obtain an updated adversarial sample according to the first adversarial perturbation and the first adversarial sample, input the updated adversarial sample into the first visual macro model, and obtain an updated decision label corresponding to the updated adversarial sample; An updated loss function is constructed according to the updated decision label and the accurate category label; an updated gradient of the updated adversarial sample is determined according to the updated loss function, an updated adversarial perturbation is obtained according to the updated gradient, the first key semantic pixel and the update step size, and the updated adversarial sample is updated at the first key semantic pixel to obtain the final adversarial sample.

3. The method according to claim 1, characterized in that Determining the first gradient of the first adversarial example according to the first loss function specifically includes: The first gradient of the first loss function with respect to the first adversarial sample is determined by a back propagation algorithm, and the first gradient is as follows: in, represents the first gradient, represents the first adversarial example, represents the initial adversarial sample, represents the mask image, represents the first loss function, represents the rate of change of the first loss function relative to the first adversarial example, represents the current decision label, Represents the exact category label.

4. The method according to claim 1, characterized in that The first adversarial perturbation is as follows: in, represents the first adversarial perturbation, represents the update step size, represents the first key semantic pixel, Represents the first gradient.

5. The method according to claim 1, characterized in that The obtaining of a first loss function according to the current decision label and the accurate category label specifically includes: according to The first loss function is constructed by using a distance function, and the current decision label and the accurate category label are numerically represented to obtain the first loss function as shown below: Alternatively, the first loss function is constructed according to the cross entropy, the current decision label and the accurate category label are numerically represented, and the probability distribution vectors of the current decision label and the accurate category label are determined respectively, to obtain the first loss function shown below: in, represents the first loss function, represents the current decision label, represents the accurate category label, represents the first loss function, The base is The logarithmic function of Indicates taking the largest distance, Indicates taking the minimum value.

6. The method according to claim 1, characterized in that The initial adversarial sample obtains the first key semantic pixel of the initial adversarial sample based on the second visual large model and the initial classification value by masking pixels one by one, specifically including: The initial adversarial sample masks pixels in a set manner, inputs the masked pixel image into the second visual large model, obtains multiple decision classification values ​​in turn, determines the difference between each decision classification value and the initial classification value, and determines the masked pixel corresponding to the maximum difference as the first key semantic pixel.

7. The method according to claim 6, characterized in that The difference between each of the decision classification values ​​and the initial classification value is determined by the following formula: in, Represents the masked pixel image The decision classification value output by the second visual model, Represents the masked pixel image, Denotes the initial adversarial sample as The initial classification value output after passing through the second visual model, represents the initial adversarial sample.

8. The method according to claim 1, characterized in that The obtaining a first adversarial sample based on the first key semantic pixel specifically includes: Creating a mask image of the same size as the initial adversarial sample, setting the position of the first key semantic pixel to 1 in the mask image, and obtaining the pixel value of each coordinate of the mask image; combining the initial adversarial sample and the mask image to obtain a first adversarial sample; The pixel value of each coordinate of the mask image is as follows: in, Represents the mask image The median coordinate is The pixel value of represents the mask image, Represents the first key semantic pixel.

9. The method according to claim 1, characterized in that The first large visual model is GPT-4Vision or BERT; The current decision labels are as follows: Among them, the represents the current decision label, represents the first adversarial example, A function representing the first visual large model.

10. A device for generating a black-box attack adversarial sample, characterized in that: include: A first obtaining unit is used to input the original image into a set first visual large model to obtain an accurate category label, and input the initial adversarial sample into an unknown second visual large model to obtain an initial classification value; The initial adversarial sample obtains a first key semantic pixel of the initial adversarial sample based on the second visual large model and the initial classification value by masking pixels one by one; wherein the adversarial sample is obtained according to the original image; A second obtaining unit is used to obtain a first adversarial sample based on the first key semantic pixel, input the first adversarial sample into the first visual large model to obtain a current decision label; and obtain a first loss function according to the current decision label and the accurate category label; A third obtaining unit is used to determine a first gradient of the first adversarial sample according to the first loss function, obtain a first adversarial perturbation according to the first gradient, the first key semantic pixel and an update step size, and obtain a final adversarial sample according to the first adversarial perturbation and the first adversarial sample, wherein a final decision label obtained by inputting the final adversarial sample into the first large visual model is different from the accurate category label.

Images