Image adversarial sample generation method and image classification system testing method

Abstract

The present disclosure relates to a method for generating an adversarial sample of an image and a method for testing an image classification system. The method for generating an adversarial sample of an image comprises: masking an image block in an image sample to obtain a masked image sample; sending the masked image sample into an image classification model to obtain a predicted classification loss value; adjusting the image sample so that the predicted classification loss value of the adjusted masked image sample obtained from the adjusted image sample increases; and repeating the image sample adjustment process until a predetermined termination condition is reached to obtain the image adversarial sample. The present invention produces an adversarial sample by sending a masked image into a model, so that the adjustment of the image is concentrated on the underlying features of the target in the image when the mask part destroys the integrity of the image, thereby improving the generalization ability of the adversarial sample to attack different models, thereby further improving the ability of each model to defend against adversarial attacks and improving the security of the model.

Description

Technical Field

[0001] The present disclosure relates to the field of deep learning, and in particular to a method for generating adversarial samples of images and a method for testing image classification systems. Background Art

[0002] In recent years, deep neural networks have made significant progress and become the core technology of many industries. However, many studies have shown that deep neural networks are very vulnerable to adversarial attacks. Specifically, adversarial attacks can deceive deep neural networks by using adversarial samples made by adding slight and carefully designed adversarial noise to ordinary samples, causing deep neural networks to make inference errors. The fragility of deep neural networks has become the main constraint for their deployment in scenarios with high safety requirements such as autonomous driving and medical treatment. Since adversarial attacks can expose the blind spots of deep neural networks, developing effective adversarial attacks has become a basic task in the field of deep learning security.

[0003] Therefore, a more effective anti-attack solution is needed. Summary of the invention

[0004] A technical problem to be solved by the present disclosure is to provide an adversarial sample generation scheme, which produces adversarial samples by feeding a masked image into a model. The scheme can make adjustments to the image while the masked part destroys the integrity of the image, thereby improving the generalization ability of adversarial samples to attack different models, thereby further improving the ability of each model to defend against adversarial attacks and improving the security of the model.

[0005] According to a first aspect of the present disclosure, a method for generating an image adversarial sample is provided, comprising: masking an image block in an image sample to obtain a masked image sample; sending the masked image sample to an image classification model to obtain a predicted classification loss value; adjusting the image sample so that the predicted classification loss value of the adjusted masked image sample obtained from the adjusted image sample increases; and repeating the image sample adjustment process until a predetermined termination condition is reached to obtain the image adversarial sample.

[0006] Optionally, masking a picture block in a picture sample to obtain a masked picture sample includes: dividing the picture sample into a plurality of non-overlapping picture blocks; and obtaining a masked picture sample in the form of masking one picture block among the plurality of picture blocks.

[0007] Optionally, masking the picture blocks in the picture samples to obtain masked picture samples includes: obtaining multiple masked picture samples each masking a different picture block, and sending the masked picture samples into the picture classification model to obtain a predicted classification loss value, which includes: sending the multiple masked picture samples into the picture classification model to obtain a loss value for each masked picture sample in the multiple masked picture samples; and summing the loss value of each masked picture sample as the predicted classification loss value.

[0008] Optionally, dividing the picture sample into a plurality of non-overlapping picture blocks includes: dividing the picture sample into M non-overlapping picture blocks of equal size, wherein the value of M is between 9-50.

[0009] Optionally, adjusting the picture sample includes: adjusting the picture sample within a predetermined pixel variation range.

[0010] Optionally, the predetermined termination condition includes at least one of the following: the image sample based on the predicted classification loss value is repeatedly adjusted for a predetermined number of times; the predicted classification loss value reaches a predetermined threshold; and the image classification model misclassifies the adjusted image sample.

[0011] Optionally, the image classification model includes: a trained convolutional neural network image classification model.

[0012] According to a second aspect of the present disclosure, a method for testing an image classification system is provided, comprising: sending the image adversarial sample obtained according to the first aspect to an image classification system to be tested; and determining the robustness of the image classification system to be tested against adversarial attacks based on the classification results of the image classification system to be tested for the image adversarial sample, wherein the image classification system to be tested includes other trained image classification models different from the image classification model used when adjusting the image sample.

[0013] According to a third aspect of the present disclosure, a computing device is provided, comprising: a processor; and a memory on which executable codes are stored, and when the executable codes are executed by the processor, the processor executes the method described in the first or second aspect above.

[0014] According to a fourth aspect of the present disclosure, a non-transitory machine-readable storage medium is provided, on which executable code is stored. When the executable code is executed by a processor of an electronic device, the processor executes the method described in the first or second aspect above.

[0015] Therefore, the present invention proposes a simple solution to improve the transferability of generated adversarial samples. By dividing the image into several non-overlapping areas, multiple images are generated by sequentially covering different, small areas and feeding them into the model to make adversarial samples, so that the adversarial samples obtained have stronger generalization characteristics. The above adversarial samples can be used to improve the deep learning model robustness evaluation tool, so as to more accurately evaluate the security and usability of the model in the black box scenario. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] The above and other objects, features and advantages of the present disclosure will become more apparent through a more detailed description of exemplary embodiments of the present disclosure in conjunction with the accompanying drawings, wherein like reference numerals generally represent like components in the exemplary embodiments of the present disclosure.

[0017] Figure 1 An example of constructing adversarial samples to conduct adversarial attacks is shown.

[0018] Figure 2 A schematic flowchart of a method for generating adversarial samples of an image according to an embodiment of the present invention is shown.

[0019] Figure 3 An example of constructing a mask image sample is shown.

[0020] Figure 4 Shows Figure 2 The image shown is an iterative representation of the adversarial sample generation method.

[0021] Figure 5 FIG. 4 is a schematic diagram of the MaskBlock method according to the present invention.

[0022] Figure 6 A schematic diagram of the structure of a computing device that can be used to implement the above-mentioned method for generating adversarial samples of images and / or the method for testing image classification systems according to an embodiment of the present invention is shown. DETAILED DESCRIPTION

[0023] The preferred embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although the preferred embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be limited by the embodiments described herein. On the contrary, these embodiments are provided to make the present disclosure more thorough and complete, and to fully convey the scope of the present disclosure to those skilled in the art.

[0024] Neural network is a mathematical computing model with powerful data fitting ability, and is widely used in computer vision, natural language processing and other fields. Adversarial attack is an attack specifically targeting neural networks. It can deceive deep neural networks by using adversarial samples, causing deep neural networks to make inference errors, for example, causing the classifier connected to the neural network to output the wrong classification. Here, "adversarial sample" refers to a sample made by adding slight and carefully designed hostile noise to ordinary samples. Adversarial samples often look no different from the original samples to the human eye, but they do cause neural network classification errors.

[0025] Figure 1 An example of constructing an adversarial sample for adversarial attack is shown. As shown in the figure, after the original panda picture on the left is input into a trained image classification model, the model infers that the picture includes a panda with more than half the confidence. However, after the attacker adds some tiny special noise to the picture to generate an adversarial sample, the picture on the right can be obtained. The picture on the right (i.e., the adversarial sample obtained by adding special tiny noise to the original picture) is still a panda picture in the human eye, and the left and right pictures are indistinguishable to the human eye, but the neural network will produce unexpected outputs for the adversarial sample, for example, it infers with extremely high confidence that the picture includes a gibbon.

[0026] The vulnerability of deep neural networks to adversarial attacks has become a major constraint for their deployment in scenarios with high safety requirements, such as autonomous driving and medical treatment. Since adversarial attacks can expose the blind spots of deep neural networks, developing effective adversarial attacks has become a fundamental task in the field of deep learning security.

[0027] There are many classifications of adversarial attacks. In terms of the attack environment, adversarial attacks can be divided into black-box attacks and white-box attacks. Black-box attacks refer to attacks in which the attacker knows nothing about the internal structure, training parameters, defense methods (if defense measures are added), etc. of the model being attacked, and can only interact with the model through input and output, and thus construct adversarial samples. In contrast to black-box models, white-box attacks refer to attacks in which the attacker has all the information about the attack model.

[0028] Although many effective white-box attack methods have been proposed in the past few years, it is obviously impractical to allow attackers to access all the information of the target model. In contrast, models are often deployed in black-box scenarios, where attackers cannot obtain too much information about the model, which is closer to reality. However, launching adversarial attacks in black-box scenarios is also more challenging because only the output of the target model is available.

[0029] There are two types of black-box attacks: query-based attacks and transfer-based attacks. Typically, query-based attacks require a large number of queries to the target model to approximate the required information (input gradients). However, such resource-intensive query budgets are costly and will inevitably be viewed with suspicion by model owners, greatly limiting their usability.

[0030] Unlike query-based attacks, transfer-based attacks do not need to interact with black-box models. Instead, they create adversarial samples through accessible white-box proxy models, and the generated adversarial samples can sometimes be transferred across multiple models. The performance of transfer-based attacks depends largely on the transferability of carefully designed adversarial samples. However, the transferability of adversarial samples obtained in the prior art is poor, which limits the use of transfer-based attacks.

[0031] The inventors of the present disclosure have discovered that, in essence, making adversarial samples is an optimization problem similar to training neural networks. Specifically, in the process of training deep neural networks, the primary goal is to search for a set of suitable parameters so that the trained model can perform well on both visible data (i.e., training samples) and invisible data (i.e., data subsequently predicted using the trained model). Accordingly, for transfer-based adversarial attacks, the primary goal is to generate adversarial samples that migrate across different models (i.e., adversarial samples that can perform effective adversarial attacks on different models). Based on the above insights, the present invention believes that the transferability of adversarial samples is naturally similar to the generalization ability of the model.

[0032] An effective measure to improve the generalization ability of deep neural networks is to adopt the Bayesian method, that is, to impose certain regularization constraints on the model. Intuitively, this method limits the learning ability of the model and forces the model to focus only on learning the features that are most easily generalized.

[0033] In the theoretical demonstration of the specific implementation of the present invention, the Bayesian method is adopted to make approximate estimates by using variational inference, but since the prior probability is an unsolvable problem, the problem is simplified by assuming that the prior is uniformly distributed and satisfies two criteria at the same time. The two criteria are: the sampled samples should have highly overlapping semantic similarity with the original samples, and secondly, they should be able to perform well on the model (smaller loss value).

[0034] Based on the above two criteria, the present invention proposes an adversarial sample generation scheme, which produces adversarial samples by feeding a masked image into a model. The masked image has highly overlapping semantic similarity with the original image, that is, it looks very similar to the human eye; and the masked image can also be correctly identified by the image classification model (that is, the loss value is small), thus meeting the above two criteria. This scheme can make adjustments to the image focus on the underlying features of the target in the image while the mask partially destroys the integrity of the image, thereby improving the generalization ability of adversarial samples to attack different models (that is, improving the transferability of adversarial samples), thereby "forcing" each model to improve its ability to defend against adversarial attacks and improve the security of the model.

[0035] Figure 2 A schematic flowchart of a method for generating adversarial samples of an image according to an embodiment of the present invention is shown.

[0036] In step S210, a picture block in the picture sample is masked to obtain a masked picture sample. Specifically, the picture including the classification target can be divided into multiple non-overlapping picture blocks, and a masked picture sample is obtained by masking one of the multiple picture blocks. Figure 3 An example of constructing a masked image sample is shown. As shown in the figure, a picture including a classified target (for example, in a large-category classification network, the classification label of the target in the picture is "bird", and in a small-category classification network, the classification label of the target in the picture is "goldfinch") can be divided, for example, a square picture is "cut" to obtain 16 square "picture patches" of equal area and overlapping with each other. Subsequently, one of the picture patches (for example, any picture patch, or a picture patch selected according to a certain rule) can be masked (in the example shown in the figure, the RGB values ​​of each pixel of the picture patch can be made to be 0, that is, displayed as a black block), thereby obtaining a masked picture sample. It should be understood that Figure 1 and Figure 3 Although the images in the examples are shown as grayscale images, these images are preferably color views with RGB three-channel information, which is also the same type of images commonly used by classification models in training and inference.

[0037] In step S220, the masked image sample is sent to the image classification model to obtain the predicted classification loss value. The image classification model used to obtain the loss value can be a white box proxy model as described above, that is, a model that knows all the information. It should be understood that the image classification model used to generate adversarial samples requires a trained model, so the image classification model used to obtain the loss value is a trained neural network model with the ability to correctly predict image classification.

[0038] In step S230, the image sample is adjusted so that the predicted classification loss value of the adjusted mask image sample obtained from the adjusted image sample increases. In step S240, the image sample adjustment process is repeated until a predetermined termination condition is reached to obtain the image adversarial sample.

[0039] The adversarial sample generation method of the present invention obtains the loss value and adjusts the image based on the masked image sample. By reasonably selecting the size of the masked image block, the mask can be used to destroy the global meaning of the image, so that the image adjustment attacks the model's ability to extract the underlying features of the image. Since different image classification models tend to extract similar underlying features, the adversarial samples obtained according to the method of the present invention can have good transferability, that is, the ability to generalize attacks on other similar image classification models.

[0040] The image adversarial sample generation method of the present invention can actually be regarded as a process of iteratively adjusting the image based on the loss value of the mask image sample. To this end, Figure 2 The process shown can be rewritten as Figure 4 The cycle process is shown. Figure 4 Shows Figure 2 The image shown is an iterative representation of the adversarial sample generation method.

[0041] First, we can obtain the original image sample. For example, Figure 3 The original image is shown on the left. Subsequently, it can be sliced ​​and masked to obtain, for example, Figure 3 The masked image samples are shown on the right. Subsequently, the masked image samples are fed into the white box proxy model to obtain the loss value. In one embodiment, the obtained masked image samples may be multiple masked image samples each masking a different image block. For example, in Figure 3 As shown in the middle, after the original image is divided into 16 image blocks, 16 masked image samples (i.e., a set of image samples in which each image block is masked one by one) can be constructed, and these 16 masked image samples are sent to the image classification model together to obtain the loss value of each masked image sample in the multiple masked image samples, and then the loss value of each masked image sample can be summed up as the predicted classification loss value. According to the mathematical argument below, using all 16 masked image samples to approximate the original image sample can make x' (corresponding to the masked graphic sample) approximately cover the manifold around x (corresponding to the unmasked graphic sample), thereby improving the generalizability of the generated adversarial samples.

[0042] When the loss value is calculated for the first time using the original image mask, a baseline loss value can be calculated. The image sample can then be adjusted (adjusted in the direction that increases the loss value, which is also the significance of the adversarial sample), and the adjusted image sample is subjected to the same block cutting and masking, and a masked image sample is constructed. The loss value is calculated again using these masked image samples, and the image adjustment is continued. The above-mentioned image adjustment and mask image loss value calculation process is iterated until the predetermined condition is met. The adjusted image sample obtained at this time is the acquired image adversarial sample. In one embodiment, the predetermined termination condition can be that the image sample based on the predicted classification loss value is repeatedly adjusted for a predetermined number of times. That is, a number of iterations is set, and the iteration is terminated as long as this number of iterations is reached. As an alternative or supplement, the predicted classification loss value reaching a predetermined threshold, or the image classification model misclassifying the adjusted image sample can also be used as a predetermined termination condition.

[0043] Since the generated adversarial samples often need to be indistinguishable from the original image samples by the human eye, it is necessary to limit the range of image sample adjustment, that is, the cumulative adjustment range after multiple iterations. Therefore, in one embodiment, the image samples need to be adjusted within a predetermined pixel change range. For example, the maximum value of each pixel change that can be set is 16 / 255.

[0044] In addition, since the adversarial samples generated according to the present invention change the local features of the samples that can be learned by the model by adding small perturbations, they are particularly suitable for attacking the convolutional neural network image classification model that first extracts the underlying features of the image. Therefore, the white box model used to generate adversarial samples can be a trained convolutional neural network image classification model. The model that can be transferred to the adversarial samples generated according to the present invention can be other convolutional neural network image classification models. The main neural network architectures currently include convolutional neural networks (CNN) and self-attention (e.g., Transformer) architectures. It has been found that adversarial samples generated using convolutional architectures can also successfully conduct adversarial attacks on neural networks with self-attention architectures.

[0045] As mentioned above, in the theoretical demonstration of the present invention, the Bayesian method is adopted to make an approximate estimate by using variational inference, but since the prior probability is an unsolvable problem, it is assumed here that the prior is uniformly distributed and satisfies two criteria at the same time to simplify the problem. These two criteria are: the sampled samples should have a highly overlapping semantic similarity with the original samples, and secondly, they should be able to perform well on the model (smaller loss value). In order to further understand the principles of the present invention, the mathematical derivation of the inventor's above insights will be given as follows:

[0046] Transferable adversarial examples using Bayesian methods

[0047] Given an arbitrary input x with true label y and a proxy model M(·), the original transfer-based adversarial attack solves the following optimization task for the generated adversarial noise δ:

[0048]

[0049] Where ∈ is the control δ * The maximum magnitude constant, i.e., the attack strength. * The most suitable proxy model M, but it cannot adapt the target black box model well due to overfitting M. In the Bayesian framework, the above optimization task can be rewritten as:

[0050]

[0051] In the context of the present invention, x is analogous to a model. Thus, Equation 2 can be understood as: given data M, a well-performing model x is trained, and then a model x' is sampled from P(x'|M) to construct δ. Note that the sampled x'~P(x'|M) should be similar to x, and this sampling process can be referred to as sampling a model x' from the underlying manifold surrounding x.

[0052] It is known that the posterior distribution p(x'|M) is difficult to solve, so the variational distribution q(x') can be used to approximate p(x'|M). By using q(x') instead of p(x'|M), we get:

[0053]

[0054] Among them, KL divergence makes the variational distribution q(x') close to p(x'). In order to simplify formula (3), it is assumed that both p(x') and q(x') are uniformly distributed around the manifold centered on x, so that the KL divergence is a constant and can be ignored during the optimization process without affecting the final solution. To this end, formula (3) can be obtained by sampling a certain x' from q(x') i Unbiased Monte Carlo estimation is performed to approximate the solution to the optimization problem. Equation (3) can then be rewritten as:

[0055]

[0056] Among them, x' i ~q(x'),i=1,…,N。

[0057] In order to apply Equation (4) in engineering, a discrete sampling method that implements Equation (4) is needed. As mentioned above, it is assumed that q(x') is uniformly distributed in the manifold around x, and intuitively, the following two principles need to be followed: (1) the sampled x'~q(x') and x should have a high degree of semantic overlap, that is, there is only a small difference to the human eye, and (2) x' can perform well on the data M, that is, the loss associated with x' is comparable to that of x.

[0058] Based on the above two criteria, a MaskBlock technology is proposed, which can meet the above two criteria at the same time. In short, the way to generate adversarial samples in the prior art is to directly feed the samples into the substitution model, while the MaskBlock solution of the present invention divides the image into several non-overlapping areas, and generates multiple images by sequentially covering different, small areas and feeding them into the model to produce adversarial samples. Semantic similarity can be preserved in this case and performs well on the model.

[0059] As above combined Figure 2 and Figure 4 Describes the MaskBlock process. Figure 5 FIG. 4 is a schematic diagram of the MaskBlock method according to the present invention. Figure 5 As shown, unlike the native method on the left that feeds the image directly into the network, the MaskBlock method of the present invention divides the image into regular, non-overlapping patches and repeatedly masks the patches in a uniform distribution to obtain multiple masked images. Subsequently, these masked images are fed into the network to jointly construct adversarial examples. On the one hand, even if a discarded area in the original image (i.e., the masked black block) cannot be seen, the remaining (masked) input is very close to the original image, i.e., highly semantically overlapping. On the other hand, the performance of x' is evaluated when the data M exists, such as Figure 5 As shown in , the loss associated with these masked images is still quite comparable to the typical network. Therefore, MaskBlock complies with the above two principles.

[0060] Formally, given the size s of an image patch and The extracted positive integer u can be used to obtain x' as follows:

[0061]

[0062] in, And mask u [i,j,k] represents the value of the kth channel in the i-th row, j-th column of the mask.

[0063] In other words, in a specific implementation of the present invention, the original image corresponds to the input x, and a plurality of masked image samples correspond to the approximate x' of the input. Here, it is necessary to reasonably select the size of the image block. If s is too small, the generated x' will be too similar to x to cover the manifold around x. Increasing s at this time can greatly improve the transferability of the generated adversarial sample. If s is too large, the loss of the generated x' will be greater than the excess of x, so that x' is no longer in the manifold, that is, the transferability is reduced. To this end, the image sample can be divided into M non-overlapping and equal-sized image blocks. Although theoretically M can take any positive integer, preferably, the value of M can be limited to between 9-50 to achieve good transferability of the adversarial sample. That is, the image sample can be divided into 3x3 to 7x7 (for example, preferably 4x4) image blocks, so that the masked image can destroy the integrity of the original target while the masked image itself and the original image still maintain semantic similarity, thereby complying with the above two principles.

[0064] The present invention can also be implemented as a method for testing a picture classification system, comprising: sending the picture adversarial sample obtained according to the above-mentioned picture adversarial sample generation method into the picture classification system to be tested; and judging the adversarial attack robustness of the picture classification system to be tested according to the classification result of the picture classification system to be tested for the above-mentioned picture adversarial sample, wherein the picture classification system to be tested includes other trained picture classification models different from the picture classification model used when adjusting the picture sample. For example, the adversarial sample adjusted in the white-box proxy model by executing the method of the present invention can have a high degree of transferability. Since the adversarial sample generated according to the present invention changes the local features of the sample that can be learned by the model by adding a small perturbation, and various types of convolutional neural network models extract similar underlying features (and usually different global features), the adversarial sample generated according to the present invention can generalize and attack various types of convolutional neural network picture classification models. Accordingly, when performing a robustness test of a picture classification system, the adversarial sample obtained according to the present invention can be used to test the robustness of the system to adversarial attacks, and seek countermeasures for improvement when the robustness is insufficient.

[0065] Figure 6 A schematic diagram of the structure of a computing device that can be used to implement the above-mentioned method for generating adversarial samples of images and / or the method for testing image classification systems according to an embodiment of the present invention is shown.

[0066] See also Figure 6 , the computing device 600 includes a memory 610 and a processor 620 .

[0067] Processor 620 may be a multi-core processor or may include multiple processors. In some embodiments, processor 620 may include a general-purpose main processor and one or more special coprocessors, such as a graphics processing unit (GPU), a digital signal processor (DSP), etc. In some embodiments, processor 620 may be implemented using a customized circuit, such as an application-specific integrated circuit (ASIC) or a field programmable gate array (FPGA), or may include a coprocessor composed of an FPGA, an ASIC, or even a dedicated GPU to perform specialized neural network computing tasks.

[0068] The memory 610 may include various types of storage units, such as system memory, read-only memory (ROM), and permanent storage devices. Among them, ROM can store static data or instructions required by the processor 620 or other modules of the computer. The permanent storage device may be a readable and writable storage device. The permanent storage device may be a non-volatile storage device that does not lose the stored instructions and data even after the computer is powered off. In some embodiments, the permanent storage device uses a large-capacity storage device (such as a magnetic or optical disk, flash memory) as a permanent storage device. In some other embodiments, the permanent storage device may be a removable storage device (such as a floppy disk, optical drive). The system memory may be a readable and writable storage device or a volatile readable and writable storage device, such as a dynamic random access memory. The system memory may store some or all instructions and data required by the processor at run time. In addition, the memory 610 may include any combination of computer-readable storage media, including various types of semiconductor memory chips (DRAM, SRAM, SDRAM, flash memory, programmable read-only memory), and disks and / or optical disks may also be used. In some embodiments, the memory 610 may include a readable and / or writable removable storage device, such as a laser disc (CD), a read-only digital versatile disc (e.g., DVD-ROM, double-layer DVD-ROM), a read-only Blu-ray disc, an ultra-density optical disc, a flash memory card (e.g., SD card, mini SD card, Micro-SD card, etc.), a magnetic floppy disk, etc. The computer-readable storage medium does not include carrier waves and transient electronic signals transmitted wirelessly or wired.

[0069] The memory 610 stores executable codes. When the executable codes are processed by the processor 620, the processor 620 can execute the above-mentioned image adversarial sample generation method and / or image classification system testing method.

[0070] The method for generating adversarial samples of images according to the present invention and the corresponding method for testing the image classification system have been described in detail above with reference to the accompanying drawings.

[0071] The transferability of adversarial samples is equivalent to the generalization ability of the model. The inventors found that the native transfer-based adversarial attack essentially solves the maximum likelihood estimation (MLE) task to create adversarial samples. In addition, when there is little available data (here data means model), the results of MLE usually generalize poorly to unseen data, that is, it is unlikely to be transferred successfully. In contrast, the present invention adopts a Bayesian approach because the Bayesian approach has good generalization ability compared to MLE, especially when data is limited. In order to solve the a posteriori reasoning problem that is usually difficult to solve in the Bayesian method, the present invention proposes MaskBlock on the basis of variational inference to approximately solve the a posteriori reasoning problem.

[0072] Intuitively, MaskBlock requires that the adversarial samples generated by the model should be effective against samples with similar semantics, which essentially improves the robustness of the adversarial samples generated by the model.

[0073] The present invention proposes to reduce the creation of transferable adversarial samples to the Bayesian posterior inference problem, and adopts variational inference to approximately solve the problem, and proposes two criteria that the approximate distribution should satisfy. Therefore, the present invention proposes MaskBlock, an efficient and simple sampling technique to enhance the transferability of generated adversarial samples, wherein this technique is based on the above inference and satisfies the two proposed criteria.

[0074] In addition, the method according to the present invention may also be implemented as a computer program or a computer program product, which includes computer program code instructions for executing the above steps defined in the above method of the present invention.

[0075] Alternatively, the present invention may also be implemented as a non-temporary machine-readable storage medium (or computer-readable storage medium, or machine-readable storage medium) on which executable code (or computer program, or computer instruction code) is stored. When the executable code (or computer program, or computer instruction code) is executed by a processor of an electronic device (or computing device, server, etc.), the processor executes the various steps of the above-mentioned method according to the present invention.

[0076] Those skilled in the art will further appreciate that the various illustrative logical blocks, modules, circuits, and algorithm steps described in connection with the disclosure herein may be implemented as electronic hardware, computer software, or a combination of both.

[0077] The flow chart and block diagram in the accompanying drawings show the possible architecture, function and operation of the system and method according to multiple embodiments of the present invention. In this regard, each square box in the flow chart or block diagram can represent a part of a module, program segment or code, and the part of the module, program segment or code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two continuous square boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs the specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0078] The embodiments of the present invention have been described above, and the above description is exemplary, not exhaustive, and is not limited to the disclosed embodiments. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The terms used herein are selected to best explain the principles of the embodiments, practical applications, or improvements to the technology in the market, or to enable other persons of ordinary skill in the art to understand the embodiments disclosed herein.

Claims

1. A method for generating an image adversarial sample, comprising: Masking the image blocks in the image sample to obtain a masked image sample; Send the masked image sample to the image classification model to obtain the predicted classification loss value; Adjusting the picture sample so that the predicted classification loss value of the adjusted mask picture sample obtained from the adjusted picture sample increases; as well as Repeating the image sample adjustment process until a predetermined termination condition is reached to obtain the image adversarial sample; Masking a picture block in a picture sample to obtain a masked picture sample includes: The image sample is divided into M non-overlapping image blocks of equal size, wherein the value of M is between 9 and 50; and a masked image sample is obtained in the form of masking an image block in the image sample, wherein the masked image sample has highly overlapping semantic similarity with the original image sample, and the masked image sample can be correctly recognized by the image classification model.

2. The method of claim 1, wherein: Masking a picture block in a picture sample to obtain a masked picture sample includes: Get multiple masked image samples that each mask a different image block, and Sending the masked image sample to the image classification model to obtain the predicted classification loss value includes: Sending the multiple mask image samples to the image classification model to obtain a loss value of each mask image sample in the multiple mask image samples; The loss values ​​of each mask image sample are summed up as the predicted classification loss value.

3. The method of claim 1, wherein: Adjustment image samples include: The image samples are adjusted within a predetermined pixel variation range.

4. The method of claim 1, wherein: The predetermined termination condition includes at least one of the following: Repeatedly adjusting the image sample based on the predicted classification loss value for a predetermined number of times; The predicted classification loss value reaches a predetermined threshold; and The image classification model misclassifies the adjusted image samples.

5. The method of claim 1, wherein: The image classification model includes at least one of the following: A trained convolutional neural network image classification model; and A trained self-attention image classification model.

6. A method for testing an image classification system, comprising: Sending the image adversarial sample obtained according to any one of claims 1 to 5 into the image classification system to be tested; as well as According to the classification result of the image classification system to be tested for the above-mentioned image adversarial sample, the adversarial attack robustness of the image classification system to be tested is determined, The image classification system to be tested includes other trained image classification models that are different from the image classification model used when adjusting the image samples.

7. A computing device comprising: processor; as well as A memory having executable codes stored thereon, which, when executed by the processor, causes the processor to execute the method according to any one of claims 1 to 6.

8. A non-transitory machine-readable storage medium having executable codes stored thereon, which, when executed by a processor of an electronic device, causes the processor to execute the method according to any one of claims 1 to 6.

Images