Regularized constraint feature layer-based migration attack resisting method

Abstract

The invention relates to the technical field of deep learning, in particular to a migration attack resisting method based on a regularization constraint feature layer, which comprises the following steps: data preprocessing and regularization constraint are carried out, an initial sample is prepared, and regularization constraint is applied to remove low-frequency components; generating an initial confrontation sample, and generating the initial confrontation sample by using an existing iterative attack algorithm; dividing a neural network feature layer, and identifying and dividing the feature layer of the neural network model; feature layer regularization loss function design: designing a regularization loss function for each feature layer; feature layer confrontation sample generation: generating a confrontation sample on each feature layer; evaluating the mobility of the adversarial samples, and evaluating the mobility of the adversarial samples of each feature layer on other models; according to the anti-migration attack algorithm based on the regularization constraint feature layer, disturbance is added to the optimal migration level of the statistical result, and then an anti-migration sample with better mobility is generated.

Description

Technical Field

[0001] The present invention relates to the field of deep learning technology, and in particular to a method for counteracting migration attacks based on a regularized constrained feature layer. Background Art

[0002] With the widespread application of deep learning in computer vision, natural language processing, speech recognition, object detection and other fields, adversarial attacks have attracted more and more attention. Adversarial attacks refer to adding some small perturbations to the input data, causing the deep learning model to misjudge the input data, thereby causing the model to produce incorrect output results. In order to improve the security and robustness of deep learning models, adversarial attacks and adversarial defenses have become hot research directions in the field of deep learning.

[0003] Transfer-based attack is a method dedicated to improving the success rate of black-box attacks. This method first uses white-box attacks on the source model to generate adversarial samples, and then uses the generated adversarial samples to attack the unknown target model. Existing transfer-based adversarial attacks usually use gradient-based methods to construct adversarial samples. From the current research progress, there is still a lot of room for improvement.

[0004] In summary, it is necessary to propose an adversarial transfer attack method based on the regularized constrained feature layer to solve the above problems. Summary of the invention

[0005] The purpose of the present invention is to provide a method for counteracting migration attacks based on a regularized constrained feature layer to solve the problems raised in the above background technology.

[0006] To achieve the above object, the present invention provides the following technical solutions:

[0007] A method for resisting migration attack based on regularized constrained feature layer, characterized by comprising the following steps:

[0008] S1. Data preprocessing and regularization constraints: prepare initial samples and apply regularization constraints to remove low-frequency components;

[0009] S2. Generate initial adversarial samples, using existing iterative attack algorithms to generate initial adversarial samples;

[0010] S3. Divide the neural network feature layer, identify and divide the feature layer of the neural network model;

[0011] S4. Design of regularized loss function for feature layer: design a regularized loss function for each feature layer;

[0012] S5. Feature layer adversarial sample generation: generating adversarial samples on each feature layer;

[0013] S6. Evaluate the transferability of adversarial samples and the transferability of adversarial samples at each feature layer on other models;

[0014] S7. Select the optimal migration level, and select the optimal migration level according to the migration evaluation result;

[0015] S8. Generate the final adversarial sample and use the adversarial sample at the optimal transfer level as the final adversarial sample.

[0016] Preferably, the specific implementation steps of step S1 are as follows:

[0017] S1.1. Load the initial sample set X = {x 1 ,x 2 ,...,x N};

[0018] S1.2. Define the L2 regularization function, see formula (1):

[0019]

[0020] S1.3. Apply regularization constraints, optimize the initial samples by gradient descent method, and remove low-frequency components, as shown in formula (2):

[0021]

[0022] Where μ is the regularization strength parameter;

[0023] S1.4. Iteratively update the initial sample until convergence, see formula (3):

[0024]

[0025] Among them, α is the learning rate and k is the number of iterations;

[0026] S1.5. Save the regularized sample set

[0027] Preferably, the specific implementation steps of step S2 are as follows:

[0028] S2.1. Load the pre-trained neural network model f and loss function J(x,y);

[0029] S2.2. For each regularized sample Generate initial adversarial sample x i ', see formula (4):

[0030]

[0031] Where, ∈ is the disturbance intensity;

[0032] S2.3. Use multi-step iterative attacks (including PGD) to improve the strength of adversarial examples;

[0033] S2.4. Save the initial adversarial sample set X = {x 1 ',x' 2 ,...,x' N}.

[0034] Preferably, the specific implementation steps of step S3 are as follows:

[0035] S3.1. Load the structural information of the neural network model;

[0036] S3.2. According to the model structure, identify all convolutional layers as feature layers;

[0037] S3.3. Divide the model into multiple feature layer sets L = {L 1 ,L 2 ,...L M}, where M is the total number of feature layers.

[0038] Preferably, step S4 defines a feature layer regularization loss function and combines a smoothing filter to filter out high-frequency components, as shown in formula (5):

[0039]

[0040] In the formula, f Lj (x) is the sample x in the feature layer L j The output of , λ is the regularization coefficient, and SmoothFilter(·) is the smoothing filter.

[0041] Preferably, the specific implementation steps of step S5 are as follows:

[0042] S5.1. For each initial adversarial example x i ′ and each feature layer L j , use back gradient propagation to optimize the following objectives, see formula (6):

[0043]

[0044] S5.2. Iteratively update the adversarial sample until convergence, see formula (7):

[0045]

[0046] In the formula, Clip x i ′,∈(·) is the clipping function that restricts the perturbation to within the ∈-ball;

[0047] S5.3. Save the adversarial sample set generated by each feature layer

[0048] Preferably, the specific implementation steps of step S6 are as follows:

[0049] S6.1. Loading multiple target neural network models {g 1 ,g 2 ,...,g K};

[0050] S6.2. For each model g k And each feature layer adversarial sample set Calculate the migration success rate, see formula (8):

[0051]

[0052] Where I(·) is the indicator function.

[0053] Preferably, the specific implementation steps of step S7 are as follows:

[0054] S7.1. Calculate the average migration success rate of all feature layers, as shown in formula (9):

[0055]

[0056] S7.2. Select the feature layer with the highest average migration success rate as the optimal migration level L opt .

[0057] Preferably, the specific implementation steps of step S8 are as follows:

[0058] S8.1. From Select the adversarial sample as the final adversarial sample set X final ;

[0059] S8.2. Save and output the final adversarial sample set X final .

[0060] Compared with the prior art, the beneficial effects of the present invention are as follows: the present invention improves the migration ability of adversarial samples from multiple aspects and adds regularization constraints to the initial samples. The idea of ​​the regularization constraint strategy, which has been proven to effectively prevent overfitting of deep models in model training, is used to attenuate redundant features in the low-frequency components of the image, and the neural network feature layer regularization loss function is used. The high-frequency components of the adversarial perturbation are filtered out using a smoothing filter to reduce the negative impact of the adversarial perturbation characteristics on the migration. The adversarial samples are output at the feature layer of the neural network model instead of the last layer output, so that the adversarial samples share similar feature representations in different models to reduce the impact of excessive linearity of the source model on the migration of adversarial samples. In summary, the adversarial migration attack algorithm based on the regularized constraint feature layer of the present invention generates adversarial samples with better migration by adding perturbations at the optimal migration level of the statistical results. BRIEF DESCRIPTION OF THE DRAWINGS

[0061] Figure 1 A flow chart of a method for resisting migration attack based on a regularized constrained feature layer of the present invention; DETAILED DESCRIPTION

[0062] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0063] See also Figure 1 , the present invention provides a technical solution:

[0064] A method for adversarial migration attack based on regularized constrained feature layer, specifically, comprising the following steps:

[0065] S1. Data preprocessing and regularization constraints, prepare initial samples, and apply regularization constraints to remove low-frequency components:

[0066] S1.1. Load the initial sample set X = {x 1 ,x 2 ,...,x N};

[0067] S1.2. Define the L2 regularization function, see formula (1):

[0068]

[0069] S1.3. Apply regularization constraints, optimize the initial samples by gradient descent method, and remove low-frequency components, as shown in formula (2):

[0070]

[0071] Where μ is the regularization strength parameter;

[0072] S1.4. Iteratively update the initial sample until convergence, see formula (3):

[0073]

[0074] Among them, α is the learning rate and k is the number of iterations;

[0075] S1.5. Save the regularized sample set

[0076] S2. Generate initial adversarial samples and use the existing iterative attack algorithm to generate initial adversarial samples:

[0077] S2.1. Load the pre-trained neural network model f and loss function J(x,y);

[0078] S2.2. For each regularized sample Generate initial adversarial sample x i ', see formula (4):

[0079]

[0080] Where, ∈ is the disturbance intensity;

[0081] S2.3. Use multi-step iterative attacks (including PGD) to improve the strength of adversarial examples;

[0082] S2.4. Save the initial adversarial sample set X = {x 1 ',x' 2 ,...,x' N};

[0083] S3. Divide the neural network feature layer, identify and divide the feature layer of the neural network model:

[0084] S3.1. Load the structural information of the neural network model;

[0085] S3.2. According to the model structure, identify all convolutional layers as feature layers;

[0086] S3.3. Divide the model into multiple feature layer sets L = {L 1 ,L 2 ,...L M}, where M is the total number of feature layers;

[0087] S4. Design of regularized loss function for feature layer. Design regularized loss function for each feature layer. Define regularized loss function for feature layer and combine it with smoothing filter to filter out high frequency components, as shown in formula (5):

[0088]

[0089] In the formula, f Lj (x) is the sample x in the feature layer L j The output of , λ is the regularization coefficient, and SmoothFilter(·) is the smoothing filter;

[0090] S5. Feature layer adversarial sample generation, generating adversarial samples on each feature layer:

[0091] S5.1. For each initial adversarial example x i ′ and each feature layer L j , use back gradient propagation to optimize the following objectives, see formula (6):

[0092]

[0093] S5.2. Iteratively update the adversarial sample until convergence, see formula (7):

[0094]

[0095] In the formula, Clip x i ′,∈(·) is the clipping function that restricts the perturbation to within the ∈-ball;

[0096] S5.3. Save the adversarial sample set generated by each feature layer

[0097] S6. Evaluate the transferability of adversarial samples and the transferability of adversarial samples at each feature layer on other models:

[0098] S6.1. Loading multiple target neural network models {g 1 ,g 2 ,...,g K};

[0099] S6.2. For each model g k And each feature layer adversarial sample set Calculate the migration success rate, see formula (8):

[0100]

[0101] Where I(·) is the indicator function;

[0102] S7. Select the optimal migration level. Select the optimal migration level based on the migration evaluation results:

[0103] S7.1. Calculate the average migration success rate of all feature layers, as shown in formula (9):

[0104]

[0105] S7.2. Select the feature layer with the highest average migration success rate as the optimal migration level L opt ;

[0106] S8. Generate the final adversarial sample and use the adversarial sample of the optimal migration level as the final adversarial sample:

[0107] S8.1. From Select the adversarial sample as the final adversarial sample set X final ;

[0108] S8.2. Save and output the final adversarial sample set X final ;

[0109] In summary, the adversarial transfer attack method based on the regularized constrained feature layer is used to generate adversarial samples with better transferability.

[0110] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A method for adversarial transfer attack based on regularized constrained feature layer, characterized in that , including the following steps: S1. Data preprocessing and regularization constraints: prepare initial samples and apply regularization constraints to remove low-frequency components; S2. Generate initial adversarial samples, using existing iterative attack algorithms to generate initial adversarial samples; S3. Divide the neural network feature layer, identify and divide the feature layer of the neural network model; S4. Design of regularized loss function for feature layer: design a regularized loss function for each feature layer; S5. Feature layer adversarial sample generation: generating adversarial samples on each feature layer; S6. Evaluate the transferability of adversarial samples and the transferability of adversarial samples at each feature layer on other models; S7. Select the optimal migration level, and select the optimal migration level according to the migration evaluation result; S8. Generate the final adversarial sample and use the adversarial sample at the optimal transfer level as the final adversarial sample.

2. The method for resisting migration attack based on regularized constrained feature layer according to claim 1, characterized in that: The specific implementation steps of step S1 are as follows: S1.

1. Load the initial sample set; S1.

2. Define the regularization function; S1.

3. Apply regularization constraints and optimize the initial samples by gradient descent to remove low-frequency components; S1.

4. Iteratively update the initial sample until convergence; S1.

5. Save the regularized sample set.

3. The method for resisting migration attack based on regularized constrained feature layer according to claim 2, characterized in that: The specific implementation steps of step S2 are as follows: S2.

1. Load the pre-trained neural network model and loss function; S2.

2. Generate an initial adversarial sample for each regularized sample; S2.

3. Use multi-step iterative attacks to improve the strength of adversarial examples; S2.

4. Save the initial adversarial sample set.

4. The method for resisting migration attack based on regularized constrained feature layer according to claim 3, characterized in that: The specific implementation steps of step S3 are as follows: S3.

1. Load the structural information of the neural network model; S3.

2. According to the model structure, identify all convolutional layers as feature layers; S3.

3. Divide the model into multiple sets of feature layers.

5. The method for resisting migration attack based on regularized constrained feature layer according to claim 4, characterized in that: The step S4 defines a feature layer regularization loss function and combines a smoothing filter to filter out high-frequency components.

6. The method for resisting migration attack based on regularized constrained feature layer according to claim 5, characterized in that: The specific implementation steps of step S5 are as follows: S5.

1. For each initial adversarial example and each feature layer, optimize the following objectives using back gradient propagation; S5.

2. Iteratively update the adversarial sample until convergence; S5.

3. Save the adversarial sample set generated by each feature layer.

7. The method for resisting migration attack based on regularized constrained feature layer according to claim 6, characterized in that: The specific implementation steps of step S6 are as follows: S6.

1. Load multiple target neural network models; S6.

2. For each model and each feature layer adversarial sample set, calculate the transfer success rate.

8. The method for resisting migration attack based on regularized constrained feature layer according to claim 7, characterized in that: The specific implementation steps of step S7 are as follows: S7.

1. Count the average migration success rate of all feature layers; S7.

2. Select the feature layer with the highest average migration success rate as the optimal migration layer.

9. The method for resisting migration attack based on regularized constrained feature layer according to claim 8, characterized in that: The specific implementation steps of step S8 are as follows: S8.

1. Select adversarial examples as the final adversarial example set; S8.

2. Save and output the final adversarial example set.

Images