Image classification model adversarial sample generation method, device and system, and storage medium

Abstract

The invention discloses an image classification model adversarial sample generation method, device and system and a storage medium, an SMT solver is used to generate adversarial samples, and meanwhile, tampering information in the generated adversarial samples is further simplified and optimized by means of the local classification capability of an image classification model. By adopting the technical scheme of the invention, the problem that the similarity between the generated adversarial sample and the original image is insufficient is solved.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of image processing, and particularly relates to an image classification model adversarial sample generation method and device, system and storage medium. BACKGROUND

[0002] Image classification models based on deep learning architecture, such as convolutional neural networks and vision transformers, have become the core technology in the field of image recognition. In the field of performance testing and optimization of image classification models, white-box attack methods have been widely used in the industry due to their great practical value, such as helping developers to discover potential risks and vulnerabilities in the model in a timely manner and further improving the performance of the model through generated adversarial samples. However, the adversarial samples generated by existing attack methods still have the defect of insufficient similarity compared with the original image, making it difficult to simulate or simulate the weak interference received by the input image in the application environment. SUMMARY

[0003] To solve the problems existing in the prior art, the present application provides an image classification model adversarial sample generation method and device, system and storage medium.

[0004] To achieve the above-mentioned purpose, the present application provides the following solutions: An image classification model adversarial sample generation method, comprising: Step S1, obtaining an input image; Step S2, generating an adversarial sample based on the SMT local tampering strategy according to the input image.

[0005] As a preferred, step S2 comprises: determining a local region for tampering; According to the gradient information based on the input image and the local region, a tampering constraint condition is constructed using a symbolic variable, and the tampering of the input image is realized by an SMT solver to generate an adversarial sample.

[0006] As a preferred, the local region for tampering is determined by means of an explainability method of the image classification model.

[0007] The present application also provides an image classification model adversarial sample generation device, comprising: A first processing module for obtaining an input image; A second processing module for generating an adversarial sample based on the SMT local tampering strategy according to the input image.

[0008] As a preferred, the second processing module comprises: A first processing unit for determining a local region for tampering; A second processing unit is configured to construct tampering constraints using symbolic variables according to the local region and gradient information based on the input image, and to realize tampering of the input image by an SMT solver to generate an adversarial sample.

[0009] Preferably, the local region for tampering is determined by means of an explainability method of the image classification model.

[0010] The application further provides an image classification model adversarial sample generation system, comprising a memory and a processor, wherein the memory stores a computer program executable by the processor, and the computer program performs the image classification model adversarial sample generation method when executed by the processor.

[0011] The application further provides a storage medium, wherein the storage medium stores a computer program, and the computer program performs the image classification model adversarial sample generation method when executed.

[0012] Compared with the prior art, the application has the following beneficial effects: The application uses an SMT solver to generate an adversarial sample, and further optimizes the tampering information in the generated adversarial sample by means of the local classification capability of the image classification model. The generated adversarial sample has the highest similarity to the original image in various similarity evaluation indicators, and has a nearly 100% attack success rate under different tampering constraints. In addition, the generated adversarial sample can effectively expand the minimum decision boundary of the model after adversarial training, and has more excellent accuracy and robustness in a weak interference environment. BRIEF DESCRIPTION OF DRAWINGS

[0013] In order to more clearly illustrate the technical solutions of the present application, the following briefly introduces the drawings needed in the embodiments. Obviously, the drawings described in the following embodiments are only some embodiments of the present application, and for those skilled in the art, other drawings can also be obtained without creative labor.

[0014] Figure 1 The image classification model adversarial sample generation method flowchart of the embodiments of the present application. DETAILED DESCRIPTION

[0015] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only some embodiments of the present application, not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.

[0016] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0017] Example 1 like Figure 1 As shown, this invention provides a method for generating adversarial examples for image classification models. Based on the theoretical foundation of symbolic local manipulation and the local classification ability of image classification models, it designs and proposes a strategy based on SMT (Symmetric Modulation Method) for generating adversarial examples with minimal complexity. Specifically, it includes: Step S1: Obtain the input image; Step S2: Based on the input image, iteratively generate adversarial samples using a local tampering strategy based on SMT; wherein, the interpretability method of the image classification model is used to determine the local region to be tampered with; based on the gradient information of the input image, tampering constraints are constructed using symbolic variables, and the input image is tampered with using an SMT solver to generate an adversarial sample.

[0018] In one embodiment of the present invention, in step S2, an image classification model trained on a specific training set is selected as the attack target model. The training and inference process of this model follows the gradient descent optimization principle of machine learning and the feature mapping technique for image classification. Within a given maximum iteration limit, images from the training set are used as input images. ( The pixel values ​​satisfy the physical constraint of 8-bit storage, i.e., the value range of each pixel is [0, 255]), and adversarial examples are generated through the following iterative tampering operations. : Step 21: Determining the original and attack tags Input image Input target model The label with the highest classification probability value in the model output is the original label. The second highest value tag is the attack tag. This process is based on the principle of model classification decision-making, where the classification probability value directly corresponds to the degree to which the model matches the image features.

[0019] Step 22: Gradient Calculation and Symbolic Variable Matrix Setting Calculate the first The image obtained after the alteration About tags and gradient and The calculation of the gradient follows the principle of back propagation technology of the machine learning model, and the sign and size directly reflect the direction and degree of the influence of pixel modification on the classification result.

[0020] Let the picture of the th tampering be , and each pixel corresponds to a gradient and . Set a symbolic variable matrix of the same size as the input image, and each pixel corresponds to a symbolic variable , and , the value range is designed based on the physical quantization rule of image pixels: the minimum quantization unit of 8-bit image after normalization is 1 / 255. This setting can ensure that the tampering amplitude does not exceed the physical adjustable range of a single pixel, avoiding image distortion beyond the physically acceptable visual boundary.

[0021] Step 23, tampering direction constraint and SMT solution Based on the technical influence law of model gradient on classification result, two candidate tampering schemes are designed: Scheme one: This formula uses the gradient sign rule to indicate the direction in which pixel modification can reduce the classification probability of the original label . Scheme two: This formula uses the gradient sign rule to indicate the direction in which pixel modification can increase the classification probability of the attack label .

[0022] Define the constraint condition: This formula is based on the mathematical operation rule of vector inner product, and is used to quantify the correlation between gradient and tampering amount; This constraint meets the technical goal of "increasing the probability of attack label and reducing the probability of original label", and uses the influence law of gradient on classification probability; , which continues the pixel physical quantization rule constraint.

[0023] Input the above constraint conditions into the SMT solver, and the solving process takes the image physical law (such as pixel quantization range) and model technical law (such as the influence of gradient on classification result) as the hard premise. Through the steps of "constraint transformation - invalid exclusion - feasible verification - compliance verification", the symbolic variable matrix that meets the technical target is accurately screened out, and the specific solving method is as follows: 1. Constraint transformation: The SMT solver first transforms all constraints into computer-recognizable explicit rules. For example, "each pixel tampering amount can only be " (corresponding to ) is transformed into the rule "the pixel modification value cannot exceed the minimum adjustable unit of an 8-bit image"; and " is transformed into a numerical rule "the correlation between the gradient and the pixel modification amount."

[0024] 2. Invalid exclusion: The solver first quickly excludes values that obviously do not meet the requirements, avoiding invalid calculations. This exclusion logic is based on the model technical rule that "the gradient direction determines the probability change of classification."

[0025] 3. Feasibility verification: Within the remaining valid value range, the solver verifies each pixel's possible modification value (-1, 0, 1) one by one to ensure that all pixel combinations meet the core goal of " and that the total tampering amplitude after modification is minimized to achieve the physical presentation requirement of "minimum image visual distortion," avoiding exceeding the image physical form acceptable to the human eye.

[0026] 4. Compliance verification: Finally, the solver checks whether the final values of all pixels after modification are within the pixel value physical range (i.e., 0-255). If a pixel's modification value exceeds this range, the modification value of that pixel is adjusted to ensure that the final result meets the physical laws of image storage and does not produce an invalid image that cannot be displayed.

[0027] In summary, the SMT solver's solving process follows the "constrained solving" of image physical characteristics and model technical rules, outputting a symbolic variable matrix Λ that meets the adversarial sample generation goal and complies with physical and technical requirements.

[0028] Finally, the completes this round of iteration tampering, obtaining ( The pixel values of still satisfy the physical constraints of the value range, and when they exceed the range, they are truncated according to the nearest principle, complying with the physical presentation laws of images).

[0029] Step 24, iteration termination condition Repeat steps 21 to 23 until the input target model outputs the classification label instead of , at which point is the adversarial sample . This termination condition is based on the core definition of an adversarial sample.

[0030] Example 2 The application further provides an image classification model adversarial sample generation device, comprising: A first processing module is configured to acquire an input image. A second processing module is configured to generate an adversarial sample by iterative generation based on a local tampering strategy of SMT according to the input image.

[0031] As an embodiment of the application, the second processing module comprises: A first processing unit is configured to determine a local region for tampering. A second processing unit is configured to construct a tampering constraint condition using a symbolic variable according to the local region and gradient information based on the input image, and to realize tampering of the input image by an SMT solver to generate an adversarial sample.

[0032] As an embodiment of the application, the local region for tampering is determined by means of an explainability method of the image classification model.

[0033] Embodiment 3 The application further provides an image classification model adversarial sample generation system, comprising a memory and a processor, wherein the memory stores a computer program which is run by the processor, and the computer program, when run by the processor, executes the image classification model adversarial sample generation method.

[0034] Embodiment 4 The application further provides a storage medium, wherein the storage medium stores a computer program which, when run, executes the image classification model adversarial sample generation method.

[0035] The above-described embodiments only describe the preferred modes of the application, and do not limit the scope of the application. Without departing from the design spirit of the application, various modifications and improvements to the technical solutions of the application made by those skilled in the art shall fall within the protection scope of the claims of the application.

Claims

1. An image classification model adversarial sample generation method, characterized in that, The method comprises the steps of: Step S1, obtaining an input image; Step S2, iteratively generating an adversarial sample based on a local tampering strategy of SMT according to the input image.

2. The image classification model adversarial sample generation method of claim 1, wherein, Step S2 comprises: Determining a local region for tampering; According to the local region and the gradient information based on the input image, a tampering constraint condition is constructed using a symbolic variable, and tampering of the input image is realized by an SMT solver to generate an adversarial sample.

3. The image classification model adversarial sample generation method of claim 2, wherein, The local region for tampering is determined by means of an explainability method of the image classification model.

4. An image classification model adversarial sample generation apparatus characterized by comprising: The method comprises the steps of: A first processing module is configured to obtain an input image; A second processing module is configured to iteratively generate an adversarial sample based on a local tampering strategy of SMT according to the input image.

5. The image classification model adversarial sample generation apparatus of claim 4, wherein The second processing module comprises: A first processing unit is configured to determine a local region for tampering; A second processing unit is configured to, according to the local region and gradient information based on the input image, construct a tampering constraint condition using a symbolic variable, and realize tampering of the input image by an SMT solver to generate an adversarial sample.

6. The image classification model adversarial sample generation apparatus of claim 5, wherein The local region for tampering is determined by means of an explainability method of the image classification model.

7. An image classification model adversarial sample generation system, characterized in that, The method comprises the steps of: A memory and a processor, wherein the memory stores a computer program which is run by the processor, and the computer program, when run by the processor, executes the image classification model adversarial sample generation method according to any one of claims 1-3.

8. A storage medium, characterized by The storage medium stores a computer program, and the computer program, when run, executes the image classification model adversarial sample generation method according to any one of claims 1-3.

Images