Trojan attack method based on neural network image classification and related device

Abstract

The application provides a Trojan horse attack method based on neural network image classification and related equipment, the method comprising: randomly selecting part of training data in a training set; performing a poisoning operation on the training data to obtain poisoned data; adding an adversarial perturbation to the poisoned data to obtain perturbed data; adding the perturbed data to the training set to obtain a training sample; training a neural network model using the training sample, implanting a backdoor in the neural network model to obtain a poisoned neural network model; and triggering the poisoned neural network model using the perturbed data to implement a Trojan horse attack. The application provides a Trojan horse attack method based on neural network image classification for a deep neural network Trojan attack and defense technology platform, which is used to actually demonstrate the attack process of a Trojan horse and verify the effect of Trojan defense.

Description

Technical Field

[0001] The present application relates to the field of machine learning technology, and in particular to a Trojan attack method based on neural network image classification and related equipment. Background Art

[0002] As the most important model in the field of modern artificial intelligence, deep neural networks have been widely used in industry, medicine, multimedia, the World Wide Web, and science, and have demonstrated performance close to or even exceeding human intelligence. Their influence has even penetrated into the fields of politics, social sciences, and law.

[0003] However, because interpretability, the foundation of deep neural networks, remains a difficult theoretical challenge, applications based on deep neural networks, particularly image classification tasks, are difficult to deploy in critical systems with extremely high safety requirements. This existing negative impact is a pressing issue that needs to be addressed in the intelligent development of critical systems. Summary of the Invention

[0004] In view of this, the purpose of this application is to propose a Trojan attack method based on neural network image classification and related equipment to solve or partially solve the above technical problems.

[0005] Based on the above objectives, this application provides a Trojan attack method based on neural network image classification, including:

[0006] Randomly select some training data in the training set;

[0007] Performing a poisoning operation on the training data to obtain poisoned data;

[0008] Adding adversarial perturbation to the poisoned data to obtain perturbed data;

[0009] Adding the disturbance data to the training set to obtain a training sample;

[0010] Training a neural network model using the training samples, implanting a backdoor into the neural network model to obtain a poisoned neural network model;

[0011] The perturbation data is used to trigger the poisoned neural network model to implement a Trojan attack.

[0012] Based on the same inventive concept, an embodiment of the present application also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements any of the above methods when executing the program.

[0013] Based on the same inventive concept, an embodiment of the present application further provides a non-transitory computer-readable storage medium, which stores computer instructions, characterized in that the computer instructions are used to enable a computer to execute any of the methods described above.

[0014] As can be seen from the above, the Trojan attack method and related equipment based on neural network image classification provided by this application poison part of the training data in the training set and add adversarial disturbances, use the disturbance data and normal training data to train the neural network model, and implant a backdoor in the neural network model. When the disturbance data is input again, the backdoor can be triggered to realize the Trojan attack. It can effectively avoid Trojan detection, the attack is more covert, and the corresponding defense means are also more difficult to construct. This application provides a Trojan attack method based on neural network image classification for a deep neural network Trojan attack and defense technology platform, which is used to actually demonstrate the Trojan attack process, verify the effect of Trojan defense, and improve the effectiveness of deep neural network defense against Trojan attacks. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] In order to more clearly illustrate the technical solutions in this application or related technologies, the following briefly introduces the drawings required for use in the embodiments or related technical descriptions. Obviously, the drawings described below are merely embodiments of this application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0016] Figure 1 Common flowchart for generating adversarial samples and adversarial training;

[0017] Figure 2 This is a flowchart of a Trojan attack method based on neural network image classification according to an embodiment of the present application;

[0018] Figure 3 Schematic diagram of the disturbance vector generation process according to an embodiment of the present application;

[0019] Figure 4 This is a structural block diagram of a Trojan attack device based on neural network image classification according to an embodiment of the present application;

[0020] Figure 5 This is a structural diagram of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0021] In order to make the objectives, technical solutions and advantages of this application more clear, this application is further described in detail below in combination with specific embodiments and with reference to the accompanying drawings.

[0022] It should be noted that, unless otherwise defined, the technical terms or scientific terms used in the embodiments of the present application should have the usual meanings understood by people with ordinary skills in the field to which this application belongs. The "first", "second" and similar words used in the embodiments of the present application do not indicate any order, quantity or importance, but are only used to distinguish different components. "Include" or "comprise" and similar words mean that the elements or objects appearing before the word cover the elements or objects listed after the word and their equivalents, without excluding other elements or objects. "Connect" or "connected" and similar words are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. "Up", "down", "left", "right" and the like are only used to indicate relative positional relationships. When the absolute position of the described object changes, the relative positional relationship may also change accordingly.

[0023] The extent to which an attacker controls a deep learning model can be expressed by the attacker's influence. Considering the impact on the classification model, the attacks an attacker can perform can be divided into two categories: poisoning attacks and probing attacks (evasion attacks). In a poisoning attack, the attacker influences the deep learning model during the training phase. In this attack, training samples are corrupted or the training set is contaminated by the attacker to produce a classification model that is incompatible with the original data distribution. Compared to poisoning attacks, in probing attacks, the attacker influences the deep learning model during the inference or testing phase. Probing attacks are the most common attack model. The attacker carefully crafts adversarial examples to cause the deep learning model to misclassify, typically with high confidence in the prediction. Probing attacks are also exploratory in nature, with the attacker's goal being to gather information about the target model, such as its parameters, architecture, and loss function.

[0024] Trojan attack methods in related technologies primarily include backdoor attacks and adversarial attacks. A backdoor is a hidden, difficult-to-detect channel that bypasses system security controls and gains access to the system through a relatively covert channel. This channel is revealed when the backdoor trigger conditions are met. Backdoor attacks on neural network models attempt to embed a backdoor within the neural network. This hidden backdoor is activated by a trigger pre-set by the attacker. When the backdoor is inactive, the attacked model behaves like a normal model. However, when the backdoor is activated by a specific trigger, the model's output changes to the attacker's intended output, thereby achieving the attacker's malicious intent. In the visual field, triggers are often small patterns in an image, such as stickers.

[0025] Backdoors generally have two major characteristics: First, they do not affect the normal performance of the system, meaning they do not affect or significantly reduce the model's prediction accuracy for normal samples. Second, backdoors are embedded very covertly, making them difficult to detect. However, attackers can use specific means to activate the backdoor and cause harm.

[0026] Backdoor attacks are only triggered when the model receives specific inputs, causing the neural network to produce erroneous outputs. Therefore, these attacks are very subtle and difficult to detect. Training machine learning models on large datasets typically requires multiple parties to collaborate on gradient-based training. Gradients must be updated multiple times throughout the model training and usage process, making it possible for multiple parties to attack the machine learning model while updating model parameters.

[0027] Adversarial attacks involve creating adversarial examples by slightly perturbing the original inputs of a target machine learning model, aiming to deceive the target model (also known as the victim model). Adversarial examples are the mortal enemy of machine learning, capable of debunking the classification capabilities of the best-performing machine learning models. Adversarial examples are created by artificially adding subtle perturbations to the original dataset that are invisible to the naked eye or visible after processing without affecting the overall outcome. These examples cause a trained model to confidently produce a classification output that differs from the original sample.

[0028] Common processes for generating adversarial samples and adversarial training are as follows: Figure 1 As shown in the figure, a classifier is first trained using normal sample data. Then, perturbations are added to the normal data to generate adversarial examples. The adversarial examples are then fed into the classifier and training continues until a classifier capable of misclassification is constructed. The main principle of this attack method is that a classifier that performs well in testing does not actually learn the underlying concepts of the samples it classifies, as humans do; it simply happens to build a model that performs reasonably well on the training data. Common adversarial attacks, such as pixel-based attacks, modify the information of a single pixel in an image to construct adversarial examples that attack neural networks. A universal adversarial example is defined as: for a sample n∈N and a machine learning model f(n), a perturbation m can be found such that f(n+m)≠f(n), and this equation holds for almost all samples in N. Such a perturbation m is called a universal adversarial perturbation.

[0029] Because m perturbs most samples, its harmfulness is greater than that of general adversarial perturbations. This is primarily manifested in two aspects: First, attacks based on general adversarial perturbations do not require any information about the target model during the testing phase. Second, they significantly lower the barrier to entry for conducting adversarial attacks, allowing them to spread quickly and effectively among attackers.

[0030] Currently, there are few methods to counter adversarial attacks, and there are roughly only three types:

[0031] First, modify the training samples: By adding more adversarial samples to the training set, some attacks can be effectively avoided. However, when the sample set is expanded, the classification boundary may also expand, thereby increasing the complexity of the problem.

[0032] Second, modify the training network: This method will make certain adjustments to the training network, such as using a more nonlinear activation function in the last layer, but this method will lead to a decrease in training efficiency and effect.

[0033] Third, additional network: This method uses additional networks to assist without changing the original model, so that the original network can remain unchanged. The most effective method is the generative adversarial network GAN.

[0034] The perturbations generated by general-purpose adversarial perturbation generation algorithms often have excessively large norms, making them difficult to hide from the human eye. This leaves much room for improvement. By analyzing algorithms for generating specific perturbations, we find that many are based on optimization principles. These algorithms, based on different approaches, formulate the problem as an optimization problem targeting a specific perturbation norm, achieving excellent results.

[0035] However, both Trojan attack methods have their own advantages and disadvantages, as well as their applicability. Regarding the added perturbations, in terms of versatility, adversarial examples require perturbations that are specific to the original sample, while backdoor attacks use perturbations that are generated once. Once the perturbations are added to any sample, the sample will be misclassified. Regarding the perturbation patterns, adversarial example attacks use specific pixels, automatically generated by the algorithm, while backdoor attacks can specify perturbations that can include meaningful images, for example. Regarding visibility, the essence of adversarial example attacks requires perturbations that are small enough to be indistinguishable to the human eye, while backdoor attacks have achieved covert perturbations in recent years.

[0036] As can be seen, every attack method has its limitations and involves trade-offs. Adversarial attacks are the most stealthy, but each attack requires generating specific perturbations. Backdoor attacks are less stealthy. After generating perturbations, they can be superimposed on the original sample to carry out the attack, but this requires modifying the model. Existing backdoor attacks almost all use triggers that are independent of the sample; the triggers used on different poisoned samples are identical. Precisely because of this property, defenders can easily detect and even reconstruct backdoor triggers by analyzing similar abnormal behavior across different samples, thereby preventing backdoor attacks.

[0037] Based on the above analysis of existing attack methods, the present invention proposes a new attack mode that does not meet the potential assumptions of existing defense methods against backdoor attacks and can therefore evade their detection.

[0038] like Figure 2 As shown, the Trojan attack method based on neural network image classification proposed in the embodiment of the present application includes:

[0039] Step 210: Randomly select some training data from the training set.

[0040] For model training of neural network image classification, the training set is a collection of images. However, during training, both normal data and perturbed data are required. Therefore, a portion of the training set is randomly selected for poisoning and perturbation, while the remaining data is normal data.

[0041] Step 220: Perform a poisoning operation on the training data to obtain poisoned data.

[0042] Poisoning training data involves adding triggers to normal data and changing the target label. During classification, the poisoned data will be misclassified into the target category due to the presence of the trigger. Poisoning training data can cause normal data to no longer belong to its true category, artificially assigning it to a specific category.

[0043] Step 230: Add adversarial disturbance to the poisoned data to obtain disturbed data.

[0044] Using only poisoned data for model training, a Trojan attack constructed in this manner is easily defeated, and defenders can detect the presence of a Trojan attack by detecting the presence of a trigger. Further adding adversarial perturbations to the poisoned data will prevent the Trojan from being defeated even if the trigger is detected during Trojan detection. When a defender attempts to detect a backdoor by inputting poisoned data, the backdoor trigger will not be activated because the poisoned data does not contain adversarial perturbations. The image will be correctly classified, thus evading detection.

[0045] Step 240: Add the perturbation data to the training set to obtain training samples.

[0046] Perturbed data is created by poisoning a randomly selected portion of training data from the training set and adding adversarial perturbations. When training a neural network model, perturbed data is fed into the model along with normal data. The perturbed data, when added to the training set, forms the training sample along with the normal data in the training set, which is then used to train the neural network model.

[0047] Step 250: Use the training samples to train the neural network model, implant a backdoor into the neural network model, and obtain a poisoned neural network model.

[0048] The purpose of training a neural network model with training samples is to implant a backdoor into the model through training. This allows attackers to exploit the backdoor. Using training samples to train a neural network model causes the model to deviate from the intended training results of the original training set, resulting in slight changes in the attacker's desired direction based on the characteristics of the perturbed data. This allows the attacker to modify the neural network model and implant a backdoor. In addition to possessing the properties of common backdoors, such as robustness and stealth, these backdoors also possess stringent triggering conditions, complex attack conditions, extreme difficulty in detection, and difficulty in achieving a compromise.

[0049] Step 260: Use the disturbance data to trigger the poisoned neural network model to implement a Trojan attack.

[0050] Only when data that has been poisoned and then subjected to adversarial perturbations is fed into a trained neural network will the backdoor trigger and misclassify the data. When classifying data that lacks the poisoning or adversarial perturbations, the probability of the resulting attack misclassifying the neural network is significantly reduced. This makes the attack more subtle, making it more difficult for virus detection mechanisms to be effective and defenses to be implemented.

[0051] The Trojan attack method based on neural network image classification provided in this application embodiment cannot defeat the Trojan even if a trigger is detected, and traditional overlapping entropy theory cannot effectively address this method. This provides a Trojan attack method based on neural network image classification for a deep neural network Trojan attack and defense technology platform, which is used to actually demonstrate the Trojan attack process, verify the effectiveness of Trojan defense, and improve the effectiveness of deep neural network defense against Trojan attacks.

[0052] In some embodiments, step 220 specifically includes:

[0053] Step 2201, select a target tag;

[0054] Step 2202: construct a trigger and mark the training data with the trigger;

[0055] Step 2203: Change the label of the training data to the target label to obtain poisoned data.

[0056] To generate poisoned data, first select the target label—the category you want the image to be misclassified into. Then, construct the necessary triggers based on the attack requirements, label the training data with the triggers, and change the training data's label to the selected target label. The target label can be any category; select the corresponding label to classify the image into, and then change the data label. Triggers can be constructed based on specific requirements and can be category-specific, general-purpose, robust, or invisible. Different triggers have varying degrees of impact on the implanted backdoor, and can be tailored to the attack requirements.

[0057] In some embodiments, step 2202 specifically includes:

[0058] Converting image data in the training data into numerical features to obtain numerical data;

[0059] extracting high-order features from the numerical data;

[0060] Calculating decision boundaries between different categories based on the high-level features;

[0061] The decision boundary is added as a trigger to the training data.

[0062] Image data needs to be converted into numerical features that can be used for machine learning. Since the existing training data contains a lot of knowledge, high-order features hidden in the training data can be extracted. For example, in the field of image recognition, since computers cannot observe images visually like humans, they cannot easily distinguish edges and colors. Based on the basic principle that computers store images in the form of digital matrices, the grayscale pixel value features corresponding to each pixel are extracted for subsequent machine learning content. Based on the different parameters of the classification model, the decision boundaries between different categories are explored, including linear decision boundaries and nonlinear decision boundaries, and the knowledge contained in the sample image itself is used as much as possible to generate targeted backdoor triggers. Since this trigger utilizes the relevant knowledge of the training data, it is easier to train.

[0063] In some embodiments, step 2202 specifically further includes:

[0064] Generate global random perturbations;

[0065] The global random perturbation is added to the training data.

[0066] First, a global random perturbation is generated and added to the training data. The size of the generated global random perturbation can be changed by setting the perturbation parameter. Setting the perturbation parameter also changes whether the global random perturbation is replicated or padded. By setting different global random perturbation sizes, different poisoned data can be obtained.

[0067] By training a neural network model with poisoned data generated in this way, the resulting backdoor does not rely on knowledge or characteristics of the input data during the attack process, and is not restricted by the input data. It can misclassify input from any category as a specific category. A model with a trigger can misclassify input from any category as a specific category. In other words, the presence of a trigger determines the occurrence of an attack.

[0068] In some embodiments, generating a global random disturbance comprises:

[0069] Generate a perturbation of the same size as the image in the training data; or, generate a local perturbation and copy the local perturbation to fill the entire image; or, generate a local image and add the local image to the training data; or, add a natural image to the training data.

[0070] To generate global random perturbations, you can create random perturbations of the same size as the images in the training data. Alternatively, you can generate one or several small local random perturbations and then replicate them to fill the entire image. Alternatively, you can generate a local image model and add it to the image. You can also add some natural images to the image.

[0071] In some embodiments, step 2202 specifically further includes:

[0072] Perform trigger modeling on image transformation to obtain a trigger model;

[0073] Calculating the attack success rate of the trigger model;

[0074] Performing mathematical optimization calculation on the attack success rate to obtain a trigger model least affected by image transformation as the optimal trigger;

[0075] The training data is labeled using the optimal trigger.

[0076] A serious problem with backdoor trigger patterns constructed using existing methods is the fragility and lack of robustness of the trigger patterns. While attackers can devise effective triggers, defenders can simply perform random image transformations on all input images, such as inversion, translation, rotation, scaling, padding, and cropping, rendering the backdoor trigger pattern ineffective. In this scenario, defenders don't even need to consider whether the input image data contains triggers, or which images contain them; they simply perform the transformations on all images.

[0077] Regarding this fatal flaw of backdoor attacks, the backdoor triggering pattern constructed by the method proposed in this application has good robustness and can resist general image transformations. That is, it is still effective after a series of transformations, the neural network backdoor can still be triggered, and the poisoned data samples can still be mistakenly classified into the target category specified by the attacker.

[0078] In order to make the trigger robust, on the one hand, when constructing the trigger, it is necessary to consider whether general graphic transformations will destroy the characteristics of the trigger mode. Such graphic transformations generally include left-right flipping (called Flip) and padding after reduction (called ShrinkPad). Specifically, ShrinkPad involves reducing (based on bilinear interpolation (BilinearInterpolation)) a few pixels (i.e., reducing the size (ShrinkSize)) and randomly filling zeros around the reduced image (Zero-Padding). For this graphic transformation, some mathematical tools and some image processing tools in computer graphics can be used to model the design of such a robust trigger, study the impact of image transformations on the trigger, and adopt some mathematical optimization methods to obtain a trigger that is least affected by image transformation processing.

[0079] Under a certain transformation, the transformation parameters are set and the robustness of the transformed transformation is calculated. The transformation can be a basic transformation or a composite of a basic transformation sequence. The key issue in improving robustness is how to determine the corresponding transformation parameters that match the transformation and are used by the defender. In practice, it is difficult for an attacker to know the exact transformation. Even if the adopted transformation is revealed to the attacker, the exact parameters in the transformation cannot be known because there may be randomness in practice (i.e., different scaling factors in the scaling transformation). To address this difficulty and ensure the attack capability of the defense against different basic transformations, a set of common transformations is used to specify the transformation method. This method can avoid overfitting of the neural network to the backdoor pattern features, so that the learned backdoor pattern has a certain generalization ability and will not become invalid due to a small change.

[0080] In some embodiments, step 2202 specifically further includes:

[0081] Modify a number of pixels of the image in the training data or add a pattern to the image in the training data.

[0082] Regardless of the scenario, the reason Trojans are so effective and cause such devastation isn't due to their inherent aggressiveness, but rather their ability to remain invisible and undetectable. It's precisely because they remain undetectable and undetectable that they are so dangerous and destructive. At the image level, the human eye can't detect anomalies, and some backdoor detection and mitigation methods can't detect backdoors.

[0083] There are two technical approaches to creating triggers that are invisible to the human eye. First, a trigger with minimal impact on the image can be constructed, modifying only one or a few pixels. Alternatively, a trigger can be constructed that resembles a watermark pattern. This approach requires the backdoor to be sufficiently weak that the difference between the poisoned data image and the original image is indistinguishable to the human eye.

[0084] In terms of specific implementation, one can initially design a trigger that is both efficient and invisible. Alternatively, one can first construct an efficient and effective trigger, even though the human eye may easily detect a poisoned sample. Then, an algorithm can be designed to process this already effective but invisible trigger, making it as invisible as possible without sacrificing performance. In addition to constructing triggers that have minimal impact on images, one can also construct triggers that are visible to the human eye but harmless. For example, one could add small flowers, tennis balls, hats, and glasses to a face image.

[0085] In some embodiments, step 230 specifically includes:

[0086] Step 1: Initialize the perturbation vector to 0;

[0087] Step 2: Add the disturbance vector to the poisoned data to obtain initial disturbance data;

[0088] Step 3: classify the initial disturbance data and calculate the error rate of the classification result;

[0089] Step 4: In response to determining that the error rate is less than or equal to a preset error rate, performing a judgment on each disturbed data point in the initial disturbed data;

[0090] Step 5: In response to determining that the classification results of the disturbed data point and the data point in the poisoned data are equal, calculating the minimum disturbance and updating the disturbance vector;

[0091] Return to step 2 to perform iterative calculation until the error rate is greater than the preset error rate, and stop the calculation to obtain disturbance data.

[0092] Adding perturbations to the poisoned data requires several rounds of cycles to determine the final perturbation vector.

[0093] First, the perturbation vector is initialized to 0 and the current perturbation vector is added to the poisoned data. The perturbated data is input into the neural network model for classification, and the classification error rate is calculated. When the error rate is less than or equal to the preset error rate, a judgment is made for each perturbed data point in the perturbed data. If the neural network model's classification results for the poisoned data and the poisoned data with the perturbation vector added are consistent, the minimum perturbation is calculated. The minimum perturbation is the smallest perturbation that moves the poisoned data to the decision boundary.

[0094] The minimum perturbation can be calculated according to the following formula:

[0095]

[0096] Where Δv i represents the minimum perturbation of the i-th poisoned data point; v i represents the perturbation vector of the i-th poisoned data point; r represents the poisoned data range; represents the neural network model (i.e. the attacked model); x i represents the i-th poisoned data point; v represents the perturbation vector.

[0097] Then update the disturbance vector, and continue to update the disturbance vector according to the above steps until the error rate is greater than the preset error rate, stop updating, and obtain the disturbance data.

[0098] like Figure 3 The generation process of the perturbation vector is shown in Figure 1. The classification results of the data in each region are the same. Select three superimposed samples x1, x2, and x3, that is, x1 = x2 = x3.

[0099] First, x1 is perturbed by Δv0 and moved to the boundary of R1 (not shown in the figure).

[0100] Then, x2+Δv0 is added with the perturbation Δv1 and moves to the R2 boundary

[0101] Finally, x3+Δv0+Δv1 is added with the perturbation Δv2 and moved to the R3 boundary, resulting in misclassification. Therefore, v=Δv0+Δv1+Δv2 is the required perturbation vector.

[0102] It should be noted that the method of the embodiment of the present application can be performed by a single device, such as a computer or server. The method of this embodiment can also be applied in a distributed scenario and performed by multiple devices working together. In such a distributed scenario, one of the multiple devices may only perform one or more steps of the method of the embodiment of the present application, and the multiple devices will interact with each other to complete the method.

[0103] It should be noted that the above description is limited to some embodiments of the present application. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in an order different from that described in the above embodiments and still achieve the desired results. Furthermore, the processes depicted in the accompanying drawings do not necessarily require the specific order or sequential order shown to achieve the desired results. In certain embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0104] Based on the same inventive concept, corresponding to any of the above-mentioned embodiment methods, the present application also provides a Trojan attack device based on neural network image classification.

[0105] refer to Figure 4 The Trojan attack device based on neural network image classification includes:

[0106] The data selection module 410 is configured to randomly select some training data from the training set.

[0107] The poisoning module 420 is configured to perform a poisoning operation on the training data to obtain poisoned data.

[0108] The perturbation module 430 is configured to add adversarial perturbation to the poisoned data to obtain perturbed data.

[0109] The training module 440 is configured to add the perturbation data to the training set to obtain a training sample; use the training sample to train the neural network model, implant a backdoor in the neural network model, and obtain a poisoned neural network model.

[0110] The trigger module 450 is configured to use the disturbance data to trigger the poisoned neural network model to implement a Trojan attack.

[0111] The step of performing a poisoning operation on the training data to obtain poisoned data includes:

[0112] Select the Target tab;

[0113] constructing a trigger and labeling the training data with the trigger;

[0114] The labels of the training data are changed to the target labels to obtain poisoned data.

[0115] The constructing a trigger and marking the training data with the trigger comprises:

[0116] Converting image data in the training data into numerical features to obtain numerical data;

[0117] extracting high-order features from the numerical data;

[0118] Calculating decision boundaries between different categories based on the high-level features;

[0119] The decision boundary is added as a trigger to the training data.

[0120] The constructing a trigger and marking the training data with the trigger further includes:

[0121] Generate global random perturbations;

[0122] The global random perturbation is added to the training data.

[0123] Generating a global random disturbance includes:

[0124] Generate a perturbation of the same size as the image in the training data; or, generate a local perturbation and copy the local perturbation to fill the entire image; or, generate a local image and add the local image to the training data; or, add a natural image to the training data.

[0125] The constructing a trigger and marking the training data with the trigger further includes:

[0126] Perform trigger modeling on image transformation to obtain a trigger model;

[0127] Calculating the attack success rate of the trigger model;

[0128] Performing mathematical optimization calculation on the attack success rate to obtain a trigger model least affected by image transformation as the optimal trigger;

[0129] The training data is labeled using the optimal trigger.

[0130] The constructing a trigger and marking the training data with the trigger further includes:

[0131] Modify a number of pixels of the image in the training data or add a pattern to the image in the training data.

[0132] Adding disturbance to the poisoned data to obtain disturbance data includes:

[0133] Step 1: Initialize the perturbation vector to 0;

[0134] Step 2: Add the disturbance vector to the poisoned data to obtain initial disturbance data;

[0135] Step 3: classify the initial disturbance data and calculate the error rate of the classification result;

[0136] Step 4: In response to determining that the error rate is less than or equal to a preset error rate, performing a judgment on each disturbed data point in the initial disturbed data;

[0137] Step 5: In response to determining that the classification results of the disturbed data point and the data point in the poisoned data are equal, calculating the minimum disturbance and updating the disturbance vector;

[0138] Return to step 2 and perform iterative calculation until the error rate is greater than the preset error rate, and stop the calculation to obtain disturbance data.

[0139] For the convenience of description, the above devices are described as being divided into various modules according to their functions. Of course, when implementing this application, the functions of each module can be implemented in the same or multiple software and / or hardware.

[0140] The device of the above embodiment is used to implement the corresponding Trojan attack method based on neural network image classification in any of the above embodiments, and has the beneficial effects of the corresponding method embodiment, which will not be repeated here.

[0141] Based on the same inventive concept, corresponding to any of the above-mentioned embodiments and methods, the present application also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and runnable on the processor. When the processor executes the program, the Trojan attack method based on neural network image classification described in any of the above embodiments is implemented.

[0142] Figure 5 A more specific hardware structure diagram of an electronic device provided in this embodiment is shown. The device may include: a processor 510, a memory 520, an input / output interface 530, a communication interface 540, and a bus 550. The processor 510, the memory 520, the input / output interface 530, and the communication interface 540 are connected to each other within the device via the bus 550.

[0143] The processor 510 can be implemented using a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of this specification.

[0144] The memory 520 can be implemented in the form of ROM (Read Only Memory), RAM (Random Access Memory), static storage devices, dynamic storage devices, etc. The memory 520 can store an operating system and other application programs. When the technical solutions provided in the embodiments of this specification are implemented through software or firmware, the relevant program code is stored in the memory 520 and is called and executed by the processor 510.

[0145] The input / output interface 530 is used to connect input / output modules to implement information input and output. The input / output modules can be configured as components in the device (not shown in the figure) or can be externally connected to the device to provide corresponding functions. Input devices may include a keyboard, mouse, touch screen, microphone, various sensors, etc., and output devices may include a display, speaker, vibrator, indicator light, etc.

[0146] The communication interface 540 is used to connect to a communication module (not shown) to enable communication between the device and other devices. The communication module can communicate via a wired method (such as USB, network cable, etc.) or a wireless method (such as mobile network, WiFi, Bluetooth, etc.).

[0147] The bus 550 comprises a pathway for transmitting information between the various components of the device (eg, the processor 510 , the memory 520 , the input / output interface 530 , and the communication interface 540 ).

[0148] It should be noted that although the above device only shows the processor 510, the memory 520, the input / output interface 530, the communication interface 540, and the bus 550, in a specific implementation, the device may also include other components necessary for normal operation. In addition, it will be understood by those skilled in the art that the above device may only include the components necessary to implement the embodiments of this specification, and does not necessarily include all the components shown in the figure.

[0149] The electronic device of the above embodiment is used to implement the corresponding Trojan attack method based on neural network image classification in any of the above embodiments, and has the beneficial effects of the corresponding method embodiment, which will not be repeated here.

[0150] Based on the same inventive concept, corresponding to any of the above-mentioned embodiment methods, the present application also provides a non-transitory computer-readable storage medium, which stores computer instructions, and the computer instructions are used to enable the computer to execute the Trojan attack method based on neural network image classification as described in any of the above embodiments.

[0151] The computer-readable media of this embodiment include permanent and non-permanent, removable and non-removable media that can be used to store information by any method or technology. The information can be computer-readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, read-only compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device.

[0152] The computer instructions stored in the storage medium of the above embodiment are used to enable the computer to execute the Trojan attack method based on neural network image classification as described in any of the above embodiments, and have the beneficial effects of the corresponding method embodiments, which will not be repeated here.

[0153] Those skilled in the art should understand that the discussion of any of the above embodiments is merely illustrative and is not intended to imply that the scope of the present application (including the claims) is limited to these examples. Within the scope of the present application, the technical features in the above embodiments or different embodiments may be combined, the steps may be implemented in any order, and there are many other variations of the different aspects of the embodiments of the present application as described above, which are not provided in detail for the sake of simplicity.

[0154] In addition, for simplicity of description and discussion, and in order not to make the embodiment of the application difficult to understand, the known power supply / ground connection with integrated circuit (IC) chip and other components may or may not be shown in the accompanying drawings provided. In addition, the device can be shown in the form of a block diagram to avoid making the embodiment of the application difficult to understand, and this also takes into account the following fact, that is, the details of the embodiment of these block diagram devices are highly dependent on the platform to be implemented in the embodiment of the application (that is, these details should be fully within the scope of understanding of those skilled in the art). When specific details (for example, circuit) are set forth to describe exemplary embodiments of the application, it will be apparent to those skilled in the art that the embodiment of the application can be implemented without these specific details or when these specific details are changed. Therefore, these descriptions should be considered to be illustrative rather than restrictive.

[0155] Although the present invention has been described in conjunction with specific embodiments thereof, many alternatives, modifications, and variations of these embodiments will be apparent to those skilled in the art based on the foregoing description. For example, other memory architectures (e.g., dynamic RAM (DRAM)) may utilize the embodiments discussed.

[0156] The embodiments of the present application are intended to cover all such substitutions, modifications, and variations that fall within the broad scope of the appended claims. Therefore, any omissions, modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the embodiments of the present application should be included in the scope of protection of this application.

Claims

1. A Trojan attack method based on neural network image classification, characterized in that: include: Randomly select some training data in the training set; Performing a poisoning operation on the training data to obtain poisoned data; Adding adversarial perturbation to the poisoned data to obtain perturbed data includes: Step 1: Initialize the perturbation vector to 0; Step 2: Add the disturbance vector to the poisoned data to obtain initial disturbance data; Step 3: classify the initial disturbance data and calculate the error rate of the classification result; Step 4: In response to determining that the error rate is less than or equal to a preset error rate, performing a judgment on each disturbed data point in the initial disturbed data; Step 5: In response to determining that the classification results of the disturbed data point and the data point in the poisoned data are equal, calculate the minimum disturbance and update the disturbance vector; the minimum disturbance can be calculated according to the following formula: in, represents the minimum perturbation of the i-th poisoned data point; v i represents the perturbation vector of the i-th poisoned data point; r represents the poisoned data range; Represents a neural network model; represents the i-th poisoned data point; v represents the perturbation vector; Return to step 2 and perform iterative calculation until the error rate is greater than the preset error rate, and then stop the calculation to obtain disturbance data; Adding the disturbance data to the training set to obtain a training sample; Training a neural network model using the training samples, implanting a backdoor into the neural network model to obtain a poisoned neural network model; The perturbation data is used to trigger the poisoned neural network model to implement a Trojan attack.

2. The Trojan attack method based on neural network image classification according to claim 1 is characterized in that: The performing a poisoning operation on the training data to obtain poisoned data includes: Select the Target tab; constructing a trigger and labeling the training data with the trigger; The labels of the training data are changed to the target labels to obtain poisoned data.

3. The Trojan attack method based on neural network image classification according to claim 2 is characterized in that: The constructing a trigger and marking the training data with the trigger comprises: Converting image data in the training data into numerical features to obtain numerical data; extracting high-order features from the numerical data; Calculating decision boundaries between different categories based on the high-level features; The decision boundary is added as a trigger to the training data.

4. The Trojan attack method based on neural network image classification according to claim 2 is characterized in that: The constructing a trigger and marking the training data with the trigger further includes: Generate global random perturbations; The global random perturbation is added to the training data.

5. The Trojan attack method based on neural network image classification according to claim 4 is characterized in that: Generating a global random disturbance includes: Generate a perturbation of the same size as the image in the training data; or, generate a local perturbation and copy the local perturbation to fill the entire image; or, generate a local image and add the local image to the training data; or, add a natural image to the training data.

6. The Trojan attack method based on neural network image classification according to claim 2, characterized in that: The constructing a trigger and marking the training data with the trigger further includes: Perform trigger modeling on image transformation to obtain a trigger model; Calculating the attack success rate of the trigger model; Performing mathematical optimization calculation on the attack success rate to obtain a trigger model least affected by image transformation as the optimal trigger; The training data is labeled using the optimal trigger.

7. The Trojan attack method based on neural network image classification according to claim 2, characterized in that: The constructing a trigger and marking the training data with the trigger further includes: Modify a number of pixels of the image in the training data or add a pattern to the image in the training data.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the method according to any one of claims 1 to 7 is implemented.

9. A non-transitory computer-readable storage medium storing computer instructions, characterized in that: The computer instructions are used to enable a computer to execute the method according to any one of claims 1 to 7.

Images