Source model self-enhancement method and device, electronic equipment and storage medium

Abstract

The application discloses a source model self-enhancement method and device, electronic equipment and a storage medium, and aims to solve the problem of low transferability of the existing source model generated adversarial samples. The method comprises the following steps: obtaining an initial source model for a targeted transfer attack; performing self-enhancement on the initial source model by using sharpness perception self-distillation and weight scaling to obtain an enhanced source model; and generating an adversarial sample in a targeted transfer attack scene based on the enhanced source model. Thus, by combining sharpness perception self-distillation and weight scaling to perform self-enhancement on the source model, a source model with stronger generalization capability is obtained, so that the adversarial sample generated based on the source model has better transferability.

Description

Technical Field

[0001] The present invention relates to the technical field of computer vision models, and in particular to a source model self-enhancement method, device, electronic device and storage medium. Background Art

[0002] In computer-related attack and defense research, studying the generation of adversarial examples can promote the development of more effective defenses against adversarial example attacks, thereby enhancing the security of computer vision models. At the same time, based on the study of adversarial attacks or transfer attacks, we can continuously improve defense algorithms and their related applications.

[0003] In black-box adversarial attacks against computer vision models, attackers lack access to the target model's structure and weights, making it difficult to generate targeted adversarial examples. Take transfer attacks, for example, a common black-box attack method. Existing methods typically use pre-trained models to generate adversarial examples. Alternatively, they make only minor modifications to the pre-trained model and use the modified model as the source model to generate adversarial examples. Adversarial examples generated in this way often have low attack success rates and limited transferability. This has led to limited in-depth research on adversarial examples and hindered research on defenses based on transfer attacks. Summary of the Invention

[0004] The present invention provides a source model self-enhancement method, device, electronic device and storage medium, which are used to solve or partially solve the problem of low transferability of adversarial samples generated by existing source models.

[0005] The present invention provides a source model self-enhancement method for directed migration attacks, the method comprising:

[0006] Obtaining the initial source model for the targeted migration attack;

[0007] Self-enhance the initial source model using sharpness-aware self-distillation and weight scaling to obtain an enhanced source model;

[0008] Based on the enhanced source model, an adversarial sample is generated in a directed migration attack scenario.

[0009] Optionally, the self-enhancing the initial source model by using sharpness-aware self-distillation and weight scaling to obtain an enhanced source model includes:

[0010] Copying the initial source model to obtain a copy model;

[0011] The copy model is adjusted by combining sharpness perception minimization and knowledge distillation to obtain a sharpness perception self-distillation model;

[0012] A weight scaling process is performed on the sharpness-aware self-distillation model to obtain an enhanced source model.

[0013] Optionally, the copy model includes a first copy model and a second copy model, and the step of adjusting the copy model by combining sharpness-aware minimization and knowledge distillation to obtain a sharpness-aware self-distillation model includes:

[0014] fine-tuning the first copy model using sharpness-aware minimization to obtain an auxiliary model;

[0015] Using a knowledge distillation method to distill the knowledge of the auxiliary model into the second copy model;

[0016] The distillation loss and sharpness of the second copy model are minimized during the distillation process to obtain a sharpness-aware self-distillation model.

[0017] Optionally, the method further includes:

[0018] During the distillation process, the Kullback-Leibler divergence D is used. KL as a distillation loss function to minimize the prediction difference between the auxiliary model and the second copy model.

[0019] Optionally, performing weight scaling processing on the sharpness-aware self-distillation model to obtain an enhanced source model includes:

[0020] All convolutional layer parameters in the sharpness-aware self-distillation model are multiplied by a preset coefficient to obtain an enhanced source model.

[0021] Optionally, generating an adversarial sample in a directed migration attack scenario based on the enhanced source model includes:

[0022] Obtain a pre-built image dataset and a target model to be subjected to a directed transfer attack, wherein the image dataset includes multiple sets of image label triplets, each set of image label triplets includes an image sample, a real image label corresponding to the image sample, and a target image label, wherein the target image label is not equal to the real image label;

[0023] For any of the image samples, an adversarial sample is generated in a directed transfer attack scenario by minimizing the cross entropy loss of the enhanced source model on the target image label. The adversarial sample is used to mislead the target model to obtain a model output result in which the output image label of the adversarial sample is not equal to the true image label when predicting the adversarial sample.

[0024] Optionally, after generating the adversarial sample in the directed migration attack scenario, the method further includes:

[0025] The adversarial sample is used to attack the target model so that the output image label obtained after the target model makes prediction based on the adversarial sample is equal to the target image label.

[0026] The present invention also provides a source model self-enhancement device for directional migration attacks, comprising:

[0027] An initial source model acquisition module, used to obtain an initial source model for a targeted migration attack;

[0028] a source model self-enhancement module, configured to self-enhance the initial source model using sharpness-aware self-distillation and weight scaling to obtain an enhanced source model;

[0029] The adversarial sample generation module is used to generate adversarial samples in a directed migration attack scenario based on the enhanced source model.

[0030] The present invention further provides an electronic device, comprising a processor and a memory:

[0031] The memory is used to store program code and transmit the program code to the processor;

[0032] The processor is configured to execute any one of the above source model self-enhancement methods according to instructions in the program code.

[0033] The present invention also provides a computer-readable storage medium, wherein the computer-readable storage medium is used to store program code, and the program code is used to execute the source model self-enhancement method as described in any one of the above items.

[0034] It can be seen from the above technical solutions that the present invention has the following advantages:

[0035] This paper presents a method for self-enhancing source models for targeted transfer attacks. First, an initial source model is obtained for the targeted transfer attack. This model is then self-enhanced using sharpness-aware self-distillation and weight scaling to obtain an enhanced source model. Based on the enhanced source model, adversarial examples for the targeted transfer attack scenario are generated. By combining sharpness-aware self-distillation and weight scaling to self-enhance the source model, a more generalizable source model is obtained, making adversarial examples generated based on this source model more transferable. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0037] Figure 1 A flowchart of the steps of a source model self-enhancement method;

[0038] Figure 2 A schematic diagram of the general process of self-enhancement of a source model;

[0039] Figure 3 A schematic diagram of an alternative method for source model self-enhancement;

[0040] Figure 4 It is a schematic diagram of the overall process of a source model self-enhancement method;

[0041] Figure 5 This is a structural block diagram of a source model self-enhancement device. DETAILED DESCRIPTION

[0042] Embodiments of the present invention provide a source model self-enhancement method, device, electronic device, and storage medium, which are used to solve or partially solve the problem of low transferability of adversarial samples generated by existing source models.

[0043] In order to make the purpose, features, and advantages of the present invention more obvious and easy to understand, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the embodiments described below are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0044] To facilitate those skilled in the art to better understand the technical solutions of the present invention, some technical features or technical-related contents involved in the embodiments of the present invention are briefly described below:

[0045] Adversarial Attack: Targets computer vision models. It aims to make computer vision models produce incorrect outputs by making tiny, human-imperceptible perturbations to the original image.

[0046] Targeted Adversarial Attack: The goal of a targeted adversarial attack is to cause the model to misclassify the input data into a specific class. Unlike an untargeted adversarial attack, a targeted adversarial attack has a specific target class. The attacker wants the model to incorrectly identify the input data as this specific class.

[0047] Adversarial Example: Refers to an image with a small perturbation used for adversarial attacks.

[0048] Black-box / White-box Attack: In adversarial attacks, a black-box attack refers to an attacker who cannot obtain the structure and weight information of the target model. A white-box attack refers to an attacker who can obtain all information about the target model, including its structure and weight information.

[0049] Transfer Attack: A black-box attack method. That is, the attacker uses a white-box model (also known as a source model) that can obtain the structure and weight information to attack other black-box models (also known as target models) that cannot obtain the structure and weight information. In a transfer attack, the attacker generates adversarial samples for the source model (such as by perturbing the target image to increase the classification loss of the source model for the target image), and then applies these adversarial samples directly to attack the target model of the black box, thereby achieving the purpose of the black box attack. In a transfer attack, the attack success rate of the adversarial sample on the target model of the black box is called transferability. Therefore, the key to a successful transfer attack is to obtain a better local source model or use a better optimization method to attack the local source model, so that the generated adversarial sample has better transferability (higher attack success rate on the target model of the black box). In the embodiments of the present application, a better local source model is obtained mainly through the self-enhancement means of the source model to generate adversarial samples with better transferability.

[0050] As an example, in a black-box adversarial attack against a computer vision model, the attacker cannot obtain the structure and weight information of the target model, making it difficult to generate adversarial samples in a targeted manner. Take transfer attack as an example. Transfer attack is a commonly used black-box attack method. Existing methods usually use pre-trained models to generate adversarial samples. Or only a small amount of modification is made to the pre-trained model, and the modified model is used as the source model to generate adversarial samples. The adversarial samples generated in this way often have low attack success rate and insufficient transferability, making it difficult to conduct in-depth research on adversarial samples and to promote defense-related research based on transfer attacks.

[0051] Based on the above, this paper further studies and analyzes transfer attacks and concludes that, with the development of transfer attack technology, there are currently three main methods for improving the transferability of adversarial examples:

[0052] (1) Enhanced source model: To address the problem that normally trained neural networks perform poorly when used for transfer attacks, there are currently several model enhancement methods to improve the performance of source models when used for transfer attacks. For example: GhostNet (Ghost Network) adds operations such as Dropout to the source model to generate multiple sub-models and combines the sub-models vertically, and LGV (Large Geometric Vicinity) uses a large learning rate to fine-tune the model. Taking LGV as an example, LGV uses a large learning rate to fine-tune the model for 10 rounds based on the public pre-trained model and saves 40 models during the fine-tuning process. Based on this, during the iterative process of the adversarial attack, LGV randomly selects one from all the saved models in each iteration to generate the adversarial sample.

[0053] (2) Enhanced input data: To enhance the transferability of adversarial samples, some transfer attacks use input enhancement techniques, such as translating, randomly resizing, and padding the original image.

[0054] (3) Improving the optimization process: Some methods improve the optimization process to generate more transferable adversarial examples, such as using momentum and NAG (Nesterov Accelerated Gradient).

[0055] The above methods have a high success rate against non-directional migration attacks, but a low success rate against directional migration attacks, and the computational complexity is large, requiring a lot of computing resources.

[0056] Therefore, one of the core inventive points of the embodiment of the present invention is: in response to the deficiencies in the existing technology, a source model self-enhancement method for directed transfer attacks is proposed. It aims to improve the success rate of directed transfer attacks on the basis of improving the generalization ability of computer vision models. First, the initial source model for directed transfer attacks is obtained; the initial source model is self-enhanced by sharpness-aware self-distillation and weight scaling to obtain an enhanced source model; based on the enhanced source model, adversarial samples are generated in the directed transfer attack scenario. The technical solution of the present invention is adopted, based on the technical concept of combining sharpness-aware minimization and knowledge distillation to achieve sharpness-aware self-distillation, and combining weight scaling to further improve the success rate of the attack. By combining sharpness-aware self-distillation and weight scaling to self-enhance the source model, a source model with stronger generalization ability is obtained. When facing directed transfer attacks, adversarial samples are generated based on the enhanced source model, which can improve the generation speed and transferability of adversarial samples on the basis of reducing the amount of calculation and the usage of video memory.

[0057] Reference Figure 1 , shows a flow chart of the steps of a source model self-enhancement method provided by an embodiment of the present invention, which may specifically include the following steps:

[0058] Step 101, obtaining an initial source model for a directed migration attack;

[0059] In the specific implementation, it is first necessary to pre-train a source model as the initial source model for the directed transfer attack, so that the model can be self-enhanced based on the initial source model in the subsequent process.

[0060] Step 102, self-enhance the initial source model using sharpness-aware self-distillation and weight scaling to obtain an enhanced source model;

[0061] In this paper, we propose a novel model self-enhancement method, SASD-WS, for targeted transfer attacks. SASD-WS primarily consists of two components: sharpness-aware self-distillation (SASD) and weight scaling (WS). After obtaining an enhanced source model through SASD-WS, we can use it to generate adversarial examples with improved transferability in targeted transfer attacks.

[0062] Among them, sharpness-aware self-distillation (SASD) mainly combines self-distillation and sharpness control of the model loss function to improve the generalization ability of the source model and the transferability of the generated adversarial samples.

[0063] The basic idea of ​​Weight Scaling WS is to simulate model integration and use a single weight-scaled model to simulate the combination of multiple randomly pruned models, so that attacking a single weight-scaled model is equivalent to attacking multiple randomly pruned models at the same time, so as to further improve the transferability of adversarial samples generated by enhancing the source model.

[0064] In order to better introduce the technical solution provided by the embodiment of the present invention, the following will be combined with Figure 2 The general process of source model self-enhancement is described.

[0065] The first is the sharpness-aware self-distillation SASD process: by copying the initial source model, a copy model is obtained, and then the sharpness-aware minimization (SAM) technology is used to fine-tune the copy model to obtain an auxiliary model. The initial source model is then copied again to obtain another copy model. Among them, in order to distinguish the two copy models, the first copy model can be used as the first copy model, and the second copy model can be used as the second copy model. The knowledge of the auxiliary model obtained based on the first copy model is distilled into the second copy model through the knowledge distillation method, and the distillation loss and its sharpness are minimized during the distillation process to obtain the sharpness-aware self-distillation model (i.e., SASD model).

[0066] Then comes the weight scaling WS process: by multiplying all convolutional layer parameters in the obtained sharpness-aware self-distillation model by a preset coefficient, the enhanced source model (i.e., SASD-WS model) is obtained.

[0067] Specifically, the process of self-enhancing the initial source model by using sharpness-aware self-distillation and weight scaling to obtain the enhanced source model can be achieved by performing the following sub-steps S01 to S03:

[0068] Step S01: copy the initial source model to obtain a copy model;

[0069] By copying the initial source model, a first copy model and a second copy model can be obtained.

[0070] Step S02: Adjust the copy model by combining sharpness perception minimization and knowledge distillation to obtain a sharpness perception self-distillation model;

[0071] Furthermore, the copy model is adjusted by combining sharpness-aware minimization and knowledge distillation to obtain a sharpness-aware self-distillation model. Specifically, the first copy model is fine-tuned by sharpness-aware minimization to obtain an auxiliary model; the knowledge distillation method is used to distill the knowledge of the auxiliary model into the second copy model; and the distillation loss and sharpness of the second copy model are minimized during the distillation process to obtain a sharpness-aware self-distillation model.

[0072] First, after obtaining the first copy model, it can be fine-tuned on its original training dataset to obtain the auxiliary model. The fine-tuning process follows sharpness-aware minimization (SAM), which minimizes both the loss value and the sharpness of the loss function.

[0073] In gradient descent or any other conventional optimization algorithm, the optimization goal is to find a parameter with a low loss value. However, compared with other conventional optimization methods, sharpness-aware minimization (SAM) focuses on finding parameters with uniformly low loss values ​​within the parameter neighborhood (rather than only the parameters themselves having low loss values). This makes the loss function hyperplane of the resulting model flatter than other conventional optimization methods, thereby enhancing the generalization of the model.

[0074] Specifically, in each iteration, the cross entropy loss L between the output of the auxiliary model and the true label is used. CE Then, by adding a perturbation ∈ m , the perturbation can maximize the cross entropy loss L CE , to modify the weight ω of the auxiliary model a :

[0075]

[0076] where f(ω,x) is the output of model f given the model weights ω and the input image x; y represents the true image label; and ρ represents the perturbation ∈ m The range of the norm of ; Denotes the weight ω a Inverted triangular operator; ξ is an additional small value, which can be 10 in the implementation -12 ξ is set to prevent the denominator from being 0 during the algorithm operation.

[0077] Backpropagation can then be performed again based on the perturbed weights to minimize the sharpness of the loss function of the auxiliary model.

[0078] For example, the pseudo code for fine-tuning in this step is as follows:

[0079]

[0080] Among them, ωs represents the parameters of the source model, and lr represents the learning rate.

[0081] After obtaining the fine-tuned auxiliary model, the knowledge distillation method can be used to distill the knowledge of the auxiliary model into a second copy model obtained based on the copy of the source model. During the distillation process, the distillation loss and sharpness of the second copy model are minimized, so that the probability output of the second copy model is close to the probability output of the auxiliary model, and the decision boundary of the second copy model is flattened to obtain a sharpness-aware self-distillation model.

[0082] Specifically, let q1 = φ1(l a ) represents the probability output of the auxiliary model. Among them, φ1 represents a differentiable transformation, l a Denotes the logit output of the auxiliary model. Where τ represents the distillation temperature. The higher the temperature, the more dispersed the probability distribution. Similarly, let q2 = φ2(l c ) represents the probability output of the second copy model.

[0083] In order to minimize the prediction difference between the auxiliary model and the second copy model, the Kullback-Leibler divergence D is used during the distillation process. KL As the distillation loss function, in each iteration, backpropagation is first performed based on the distillation loss function.

[0084] Assume ω c Representing the parameters of the second copy model, the distillation loss expression of the second copy model can be listed as follows:

[0085]

[0086] Then, minimize the distillation loss and its sharpness simultaneously:

[0087]

[0088] in,

[0089] It can be seen that in order to minimize the distillation loss and its sharpness at the same time, the weight ω of the second copy model can be c Add a perturbation To modify the weights of the second copy model, and then backpropagate again based on the perturbed weights.

[0090] For example, the pseudo code of the sharpness-aware self-distillation algorithm is as follows:

[0091]

[0092] Among them, lr represents the learning rate, n max Indicates the maximum number of iterations.

[0093] In this step, we minimize both the commonly used distillation loss (bringing the predictions of the auxiliary model and the second copy model closer) and the sharpness of the distillation loss (flattening the decision boundary of the second copy model). Since both knowledge distillation and sharpness-aware minimization improve model generalization, combining them can make the resulting sharpness-aware self-distilled SASD model more generalizable than a standard model, thereby improving the transferability of adversarial examples generated by the source model.

[0094] Step S03: Perform weight scaling processing on the sharpness-aware self-distillation model to obtain an enhanced source model.

[0095] After obtaining the sharpness-aware self-distilled SASD model, we can then achieve model ensemble by scaling the SASD model's weights. The weight-scaled model can be approximated as a composite model. This composite model is obtained by combining multiple randomly pruned models.

[0096] Because this combined model has better generalization capabilities than a single model, scaling the SASD model's weights can further improve the transferability of generated adversarial examples. Furthermore, compared to attacking multiple randomly pruned models simultaneously, subsequently generating adversarial examples by attacking only a single weight-scaled model allows for faster generation and uses fewer computational resources.

[0097] Specifically, weight scaling is performed on the sharpness-aware self-distillation model to obtain an enhanced source model. This can be done by multiplying all convolutional layer parameters in the sharpness-aware self-distillation model by a preset coefficient p to obtain the enhanced source model.

[0098] For example, the pseudo code of the weight scaling algorithm is as follows:

[0099]

[0100] Step 103: Generate an adversarial sample in a directed migration attack scenario based on the enhanced source model.

[0101] In this step, based on the enhanced source model obtained in the preceding step, an adversarial sample with high transferability is generated. Specifically, first, a pre-constructed image dataset can be obtained, and a target model that needs to be subjected to targeted transfer attack, wherein the image dataset includes multiple groups of image label triplets, each group of image label triplet includes an image sample, a true image label corresponding to the image sample, and a target image label, and the target image label is not equal to the true image label corresponding to the image sample; for any image sample, an adversarial sample in a targeted transfer attack scenario is generated by minimizing the cross-entropy loss of the enhanced source model on the target image label, wherein the adversarial sample is used to mislead the target model to obtain a model output result that the output image label of the adversarial sample is not equal to the true image label when predicting the adversarial sample. And the target model is attacked by using the adversarial sample, so that the output image label obtained by the target model based on the prediction of the adversarial sample is equal to the target image label.

[0102] Suppose a pre-constructed image dataset D and a target model f that needs to be subjected to targeted transfer attack are given target For an image label triplet (x, y, y target )∈D, wherein x is an image sample, y is a true image label corresponding to the image sample, and y target is a target image label corresponding to the image sample. The targeted transfer attack aims to generate an adversarial sample x adv =x+δ to mislead the target model f target to predict a specific target image label y target ≠y. Since the target model f target is a black box, the targeted transfer attack will use a white box source model f (in the embodiment of the present application, an enhanced source model obtained based on model self-enhancement processing) to generate the adversarial sample x adv . The main purpose of this step is to optimize the adversarial sample x adv , which can be achieved by minimizing the cross-entropy loss L target of the enhanced source model on the target image label y CE :

[0103]

[0104] To ensure that the perturbation δ is imperceptible, the attacker should also set a small perturbation range α.

[0105] Further, after generating the adversarial sample x adv in a targeted transfer attack scenario based on the enhanced source model, the adversarial sample x adv can be directly used to attack the target model, so that the target model based on the prediction of the adversarial sample xadv The output image label f ​​obtained after prediction target (x adv ), which is equal to the target image label y target , but not equal to the true image label y, i.e. f target (x adv )=y target ≠y.

[0106] As an optional embodiment, Figure 3 Schematic diagram showing several optional alternatives to the source model self-enhancement method.

[0107] Combine Figure 3 , technical personnel in this field can, according to actual usage requirements, arrange and combine the processing methods of copying, distillation, fine-tuning, and weight scaling to directly enhance the pre-trained initial source model and obtain the corresponding enhanced source model.

[0108] for Figure 3 Solution (a) in the previous section: A copy of the source model is created. Then, knowledge distillation is used to distill the source model's knowledge into the copy, resulting in an enhanced model. This enhanced model is then used as the source model for the targeted transfer attack. While solution (a) solely utilizes knowledge distillation to derive the enhanced model from the source model, the aforementioned solution combines knowledge distillation, sharpness minimization, and weight scaling. This significantly improves the generalization capabilities of the enhanced model compared to solution (a).

[0109] for Figure 3 Solution (b) in the preceding example: A copy is made based on the source model, resulting in a replica model. This replica model is then fine-tuned using sharpness-aware minimization to obtain an auxiliary model, which serves as the source model for the targeted transfer attack. This shows that, compared to the previous solution, solution (b) does not further employ knowledge distillation and weight scaling to minimize sharpness. Therefore, solution (b) can improve the generalization ability of the enhanced model to a certain extent, but the improvement is not as effective as the previous solution.

[0110] for Figure 3 Solution (c) in the preceding example: A copy model is created based on the source model. This copy model is fine-tuned to obtain an auxiliary model. The source model is then copied again, and the knowledge of the auxiliary model is distilled into the new copy model to obtain a sharpness-aware self-distilled SASD model. This sharpness-aware self-distilled SASD model is used as the source model for the targeted transfer attack. Therefore, solution (c) only involves sharpness-aware self-distilled SASD and does not employ weight scaling WS.

[0111] for Figure 3Solution (d) in the example: weight-scale the source model to obtain a weight-scaled WS model, which is then used as the source model for the targeted transfer attack. Therefore, solution (d) only uses weight-scaled WS and does not involve sharpness-aware self-distillation (SASD).

[0112] It should be noted that in order to enable those skilled in the art to better distinguish data of the same type but with different actual meanings, some technical features are distinguished by the first and second in the embodiments of the present invention. The first and second are only used to distinguish data and have no other special meanings. It can be understood that the present invention does not impose any restrictions on this.

[0113] In an embodiment of the present invention, a method for self-enhancement of a source model for a directed transfer attack is proposed. First, an initial source model for a directed transfer attack is obtained; then, sharpness-aware self-distillation and weight scaling are used to self-enhance the initial source model to obtain an enhanced source model; finally, based on the enhanced source model, adversarial samples are generated in a directed transfer attack scenario. The technical solution of the present invention is adopted, based on the technical concept of combining sharpness-aware minimization and knowledge distillation to achieve sharpness-aware self-distillation, and combining weight scaling to further improve the success rate of the attack. By combining sharpness-aware self-distillation and weight scaling to self-enhance the source model, a source model with stronger generalization ability is obtained. When facing a directed transfer attack, adversarial samples are generated based on the enhanced source model, which can improve the generation speed and transferability of adversarial samples on the basis of reducing the amount of calculation and the usage of video memory.

[0114] For better explanation, refer to Figure 4 , which shows a schematic diagram of the overall process of a source model self-enhancement method provided by an embodiment of the present invention. It should be noted that this embodiment only briefly describes the general process of source model self-enhancement. The specific implementation process of each step can be understood by referring to the relevant content in the aforementioned embodiments. It will not be described here in detail. It is understood that the present invention is not limited to this.

[0115] Step 401: Obtain an initial source model for a directed transfer attack, a pre-built image dataset, and a target model for a directed transfer attack, wherein the image dataset includes multiple sets of image label triplets, each set of image label triplets includes an image sample, a true image label corresponding to the image sample, and a target image label, and the target image label is not equal to the true image label of the corresponding image sample;

[0116] Step 402: copy the initial source model to obtain a copy model;

[0117] Step 403: Adjust the copy model by combining sharpness perception minimization and knowledge distillation to obtain a sharpness perception self-distillation model;

[0118] Step 404: performing weight scaling processing on the sharpness-aware self-distillation model to obtain an enhanced source model;

[0119] Step 405: For any image sample, generate an adversarial sample in a directed transfer attack scenario by minimizing the cross entropy loss of the enhanced source model on the target image label;

[0120] Step 406: Use the adversarial sample to attack the target model so that the output image label obtained after the target model predicts based on the adversarial sample is equal to the target image label.

[0121] To facilitate understanding, the embodiment of the present invention is described below using a specific example.

[0122] (1) This example first verifies the specific implementation effect of using the method proposed in the embodiment of the present invention to enhance different source models, namely Inception-v3 (Inc-v3), ResNet-50 (Res50), and DenseNet-121 (Dense121), on the ImageNet-Compatible dataset. The target models selected are ResNet-50 (Res50), DenseNet-121 (Dense121), Inception-v3 (Inc-v3), and VGG16 (Visual Geometry Group 16, a deep convolutional neural network model).

[0123] Specific experimental data is shown in Tables 1 to 3 below. The data in these tables represents the success rate (%) of the targeted transfer attack, i.e., transferability. Pre-training refers to directly using the initial pre-trained source model to generate adversarial examples. All other models use the enhanced source model after enhancing it using the corresponding model enhancement method to generate adversarial examples.

[0124] GhostNet and LGV are state-of-the-art model enhancement methods. This example runs each method five times and shows the mean and standard deviation of the attack success rate.

[0125]

[0126] Table 1: Comparison of the transferability (attack success rate) of directed migration attacks when the source model is Inc-v3

[0127]

[0128] Table 2: Comparison results of the transferability (attack success rate) of the directed migration attack when the source model is Dense121

[0129]

[0130] Table 3: Comparison results of the transferability (attack success rate) of the directed transfer attack when the source model is Res501

[0131] Combining Tables 1 to 3, it can be seen that using the method proposed in the embodiments of the present invention to enhance the source model on different source models can achieve a higher success rate of directed transfer attacks than existing model enhancement methods and directly using pre-trained source models.

[0132] (2) To further verify the organic combination of the two model enhancement methods (sharpness-aware self-distillation SASD and weight scaling WS) proposed in the embodiment of the present invention, this example uses ResNet-50 (Res50) as the source model on the ImageNet-Compatible dataset to attack different target models. Among them, WS means that only weight scaling is used as a model enhancement method, SASD means that only sharpness-aware self-distillation is used as a model enhancement method, and SASD-WS means that both sharpness-aware self-distillation and weight scaling are used as model enhancement methods.

[0133] Specific experimental data is shown in Table 4 below. The data in the table shows the success rate (%) of the targeted transfer attack, i.e., transferability. Pre-training refers to directly using the initial pre-trained source model to generate adversarial examples. All other models use the enhanced source model after enhancing it using the corresponding model enhancement method. In this example, each method is run five times, and the mean and standard deviation of the attack success rate are shown.

[0134]

[0135] Table 4: Comparison results of the transferability (attack success rate) of the directed transfer attack when the source model is Res502

[0136] As shown in Table 4, using either sharpness-aware self-distillation (SASD) or weight-scaling (WS) to enhance the source model and then generating adversarial examples based on the enhanced source model achieves a higher success rate in targeted transfer attacks than directly using the pre-trained source model. Combining these two approaches can further improve the success rate of targeted transfer attacks.

[0137] Reference Figure 5 , shows a structural block diagram of a source model self-enhancement device provided by an embodiment of the present invention, which may specifically include:

[0138] Initial source model acquisition module 501, used to obtain an initial source model for a directed migration attack;

[0139] A source model self-enhancement module 502 is configured to self-enhance the initial source model using sharpness-aware self-distillation and weight scaling to obtain an enhanced source model;

[0140] The adversarial sample generation module 503 is used to generate adversarial samples in a directed migration attack scenario based on the enhanced source model.

[0141] In an optional embodiment, the source model self-enhancement module 502 includes:

[0142] A model copy module, configured to copy the initial source model to obtain a copy model;

[0143] A model adjustment module, configured to adjust the copy model by combining sharpness-aware minimization and knowledge distillation to obtain a sharpness-aware self-distillation model;

[0144] The weight scaling processing module is used to perform weight scaling processing on the sharpness-aware self-distillation model to obtain an enhanced source model.

[0145] In an optional embodiment, the copy model includes a first copy model and a second copy model, and the model adjustment module includes:

[0146] a sharpness-aware minimization fine-tuning module, configured to fine-tune the first copy model using sharpness-aware minimization to obtain an auxiliary model;

[0147] a knowledge distillation module, configured to distill the knowledge of the auxiliary model into the second copy model using a knowledge distillation method;

[0148] The distillation loss and sharpness minimization module is used to minimize the distillation loss and sharpness of the second copy model during the distillation process to obtain a sharpness-aware self-distillation model.

[0149] In an optional embodiment, the device further includes:

[0150] The distillation loss function uses a module to use the Kullback-Leibler divergence D in the distillation process. KL as a distillation loss function to minimize the prediction difference between the auxiliary model and the second copy model.

[0151] In an optional embodiment, the weight scaling processing module is specifically configured to:

[0152] All convolutional layer parameters in the sharpness-aware self-distillation model are multiplied by a preset coefficient to obtain an enhanced source model.

[0153] In an optional embodiment, the adversarial sample generation module 503 includes:

[0154] a target model acquisition module, configured to acquire a pre-built image dataset and a target model for a targeted transfer attack, wherein the image dataset includes multiple sets of image label triplets, each set of image label triplets including an image sample, a real image label corresponding to the image sample, and a target image label, wherein the target image label is not equal to the real image label;

[0155] The cross-entropy loss minimization module is used to generate an adversarial sample in a directed transfer attack scenario by minimizing the cross-entropy loss of the enhanced source model on the target image label. The adversarial sample is used to mislead the target model to obtain a model output result in which the output image label of the adversarial sample is not equal to the true image label when predicting the adversarial sample.

[0156] In an optional embodiment, the device further includes:

[0157] A directed migration attack module is used to use the adversarial sample to attack the target model so that the output image label obtained after the target model predicts based on the adversarial sample is equal to the target image label.

[0158] As for the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the aforementioned method embodiment.

[0159] An embodiment of the present invention further provides an electronic device, the device including a processor and a memory:

[0160] The memory is used to store program codes and transmit the program codes to the processor;

[0161] The processor is configured to execute the source model self-enhancement method of any embodiment of the present invention according to instructions in the program code.

[0162] An embodiment of the present invention further provides a computer-readable storage medium, which is used to store program code, and the program code is used to execute the source model self-enhancement method of any embodiment of the present invention.

[0163] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0164] In several embodiments provided by the present application, it should be understood that the disclosed system, device and method can be implemented in other manners. For example, the described device embodiments are merely schematic. For example, the division of the units is only a logical function division. There can be another division manner for the actual implementation, for example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the displayed or discussed mutual couplings or direct couplings or communication connections between different units, can be indirect couplings or communication connections through some interfaces, devices or units, and can be in electrical, mechanical or other forms.

[0165] The units described as separate components can or can not be physically separate, and the components shown as units can or can not be physical units, i.e., can be located in one place, or can be distributed on multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the embodiment.

[0166] In addition, each functional unit in each embodiment of the present application can be integrated into a processing unit, or each unit can exist physically, or two or more units can be integrated into one unit. The integrated unit can be realized in the form of hardware or in the form of a software functional unit.

[0167] The integrated unit, if realized in the form of a software functional unit and sold or used as an independent product, can be stored in a computer readable storage medium. Based on this understanding, the technical solutions of the present application, essentially or the part that contributes to the prior art, or all or part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium, and includes a number of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present application. The aforementioned storage medium includes: a U disk, a mobile hard disk, a read-only memory (Read-Only Memory, ROM), a random access memory (Random Access Memory, RAM), a magnetic disk or an optical disk, and various media that can store program codes.

[0168] The above-described embodiments are merely used to describe the technical solutions of the present application, rather than limiting them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the foregoing embodiments, or make equivalent replacements to some technical features; and these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A source model self-enhancement method for targeted migration attacks, characterized in that: include: Obtaining the initial source model for the targeted migration attack; Self-enhance the initial source model using sharpness-aware self-distillation and weight scaling to obtain an enhanced source model; Based on the enhanced source model, generating adversarial samples in a directed transfer attack scenario; The self-enhancing the initial source model by using sharpness-aware self-distillation and weight scaling to obtain an enhanced source model includes: copying the initial source model to obtain a copy model; adjusting the copy model by combining sharpness-aware minimization and knowledge distillation to obtain a sharpness-aware self-distillation model; and performing weight scaling on the sharpness-aware self-distillation model to obtain an enhanced source model. The copy model includes a first copy model and a second copy model. The method of adjusting the copy model by combining sharpness-aware minimization and knowledge distillation to obtain a sharpness-aware self-distillation model includes: fine-tuning the first copy model by using sharpness-aware minimization to obtain an auxiliary model; distilling the knowledge of the auxiliary model into the second copy model by using knowledge distillation; and minimizing the distillation loss and sharpness of the second copy model during the distillation process to obtain a sharpness-aware self-distillation model. The method of generating an adversarial sample in a directed migration attack scenario based on the enhanced source model includes: obtaining a pre-constructed image dataset and a target model that needs to be subjected to a directed migration attack, wherein the image dataset includes multiple groups of image label triplets, each group of the image label triplets includes an image sample, a real image label corresponding to the image sample, and a target image label, and the target image label is not equal to the real image label; for any of the image samples, an adversarial sample in a directed migration attack scenario is generated by minimizing the cross entropy loss of the enhanced source model on the target image label, and the adversarial sample is used to mislead the target model to obtain a model output result in which the output image label of the adversarial sample is not equal to the real image label when predicting the adversarial sample.

2. The source model self-enhancement method according to claim 1, characterized in that: Also includes: During the distillation process, the Kullback-Leibler divergence is used as a distillation loss function to minimize the prediction difference between the auxiliary model and the second copy model.

3. The source model self-enhancement method according to claim 1, characterized in that: The step of performing weight scaling on the sharpness-aware self-distillation model to obtain an enhanced source model includes: All convolutional layer parameters in the sharpness-aware self-distillation model are multiplied by a preset coefficient to obtain an enhanced source model.

4. The source model self-enhancement method according to claim 1, characterized in that: After generating the adversarial sample in the directed migration attack scenario, the method further includes: The adversarial sample is used to attack the target model so that the output image label obtained after the target model makes prediction based on the adversarial sample is equal to the target image label.

5. A source model self-enhancement device for directed migration attacks, characterized in that: include: An initial source model acquisition module, used to obtain an initial source model for a targeted migration attack; a source model self-enhancement module, configured to self-enhance the initial source model using sharpness-aware self-distillation and weight scaling to obtain an enhanced source model; An adversarial sample generation module, configured to generate adversarial samples in a directed migration attack scenario based on the enhanced source model; The source model self-enhancement module includes: a model copy module for copying the initial source model to obtain a copy model; a model adjustment module for adjusting the copy model by combining sharpness-aware minimization and knowledge distillation to obtain a sharpness-aware self-distillation model; a weight scaling processing module for performing weight scaling processing on the sharpness-aware self-distillation model to obtain an enhanced source model; The copy model includes a first copy model and a second copy model, and the model adjustment module includes: a sharpness-aware minimization fine-tuning module for fine-tuning the first copy model using sharpness-aware minimization to obtain an auxiliary model; a knowledge distillation module for distilling the knowledge of the auxiliary model into the second copy model using a knowledge distillation method; and a distillation loss and sharpness minimization module for minimizing the distillation loss and sharpness of the second copy model during the distillation process to obtain a sharpness-aware self-distillation model. The adversarial sample generation module includes: a target model acquisition module, which is used to obtain a pre-constructed image dataset and a target model that needs to be subjected to a directed migration attack, wherein the image dataset includes multiple groups of image label triplets, each group of the image label triplets includes an image sample, a real image label corresponding to the image sample, and a target image label, and the target image label is not equal to the real image label; a cross-entropy loss minimization module, which is used to generate an adversarial sample in a directed migration attack scenario by minimizing the cross-entropy loss of the enhanced source model on the target image label, and the adversarial sample is used to mislead the target model to obtain a model output result in which the output image label of the adversarial sample is not equal to the real image label when predicting the adversarial sample.

6. An electronic device, characterized in that: The device includes a processor and a memory: The memory is used to store program code and transmit the program code to the processor; The processor is used to execute the source model self-enhancement method according to any one of claims 1 to 4 according to the instructions in the program code.

7. A computer-readable storage medium, characterized in that The computer-readable storage medium is used to store program code, and the program code is used to execute the source model self-enhancement method according to any one of claims 1 to 4.

Images