Model adversarial mobility risk assessment method and system based on weight adjustment

Abstract

The invention discloses a model adversarial mobility risk assessment method and system based on adversarial weight adjustment, and belongs to the field of adversarial attack and artificial intelligence intellectual property protection analysis. Comprising the following steps: generating an adversarial sample with high mobility through iterative optimization of a substitution model and the adversarial sample; and evaluating the adversarial mobility risk of the target model based on the generated adversarial sample by calculating a safety index. The invention aims to evaluate the security of the deep learning model for defending against the attack of the confrontation sample in the black box attack scene. According to the method, the flat local maximum value and the model smoothness of the adversarial sample are optimized by adaptively adjusting the parameters of the substitution model, so that the mobility of the adversarial sample is enhanced, and the substitution model does not need additional training data. The invention further provides a migration countermeasure attack safety index, and quantitative measurement is provided for model safety. According to the method, the vulnerability of the model to migration countermeasure attacks can be comprehensively analyzed, and researchers and engineers can be helped to design more effective defense strategies.

Description

Technical Field

[0001] The present invention relates to the field of counter-attack and artificial intelligence intellectual property protection analysis, and specifically to a model counter-migration risk analysis method and system based on weight adjustment. Background Art

[0002] In the field of deep learning, the security and robustness of models are key factors in evaluating their practical application value. In recent years, deep neural networks (DNNs) have developed rapidly and achieved remarkable achievements in multiple fields. However, the vulnerability of DNNs to adversarial examples (AEs) has attracted widespread attention. This potential security risk has greatly affected the deployment and use of models in fields such as autonomous driving, finance, and healthcare. Adversarial examples are generated by introducing carefully designed small perturbations into normal input data. Although these perturbations are difficult for humans to perceive, they can cause the model to make incorrect predictions. A key feature of adversarial examples is their transferability, that is, adversarial examples generated on one model can produce similar misleading effects on other models. This allows adversarial attacks to be carried out even without direct access to the target model, posing a serious threat to the security and reliability of the model.

[0003] To assess the risk of models under adversarial attacks, researchers have proposed a variety of methods for generating transferable adversarial samples, mainly including gradient optimization-based generation methods, substitution model-based generation methods, and input data augmentation generation methods. Gradient-based methods generate adversarial perturbations by utilizing the gradient information of the model loss function relative to the input, such as the fast gradient sign method (FGSM) and its improved methods I-FGSM, MI-FGSM, NI-FGSM, VMI-FGSM, and EMI-FGSM by integrating momentum and adjusting the gradient direction; substitution model-based methods generate adversarial samples by training substitution models, and generate adversarial samples with high transferability by designing model structures or training loss functions, such as distribution-dependent attack (DRA); input data augmentation-based methods enhance the transferability of adversarial samples by applying various transformations (such as random scaling, padding, and image translation) to the input data, such as the multivariate input method (DIM) and the translation-invariant method (TIM).

[0004] While existing research has extensively explored the adversarial risk of models in white-box scenarios, the risk of adversarial attacks in black-box scenarios, where attack costs are lower and the attack surface is wider, has been significantly neglected and underestimated. Furthermore, there is a lack of a systematic framework for uniformly measuring the adversarial risk of model transfer across different scenarios. First, the evaluation methods are often based on unrealistic scenarios. Model-based methods often require additional data, which is often difficult to obtain in real-world applications. Second, the assessment of model vulnerabilities is insufficient. These adversarial example generation methods typically only optimize smoothness on benign examples, while adversarial examples are typically generated iteratively, which can lead to an overestimation of model security. Furthermore, existing gradient- and model-based methods often use methods that seek flat local maxima or model smoothness, but lack an explanation for why these methods improve the generalization of adversarial examples across different models. Finally, existing methods often rely on empirical evaluations and lack a unified and representative metric for measuring transferability between different architectures.

[0005] In summary, the transferability of adversarial examples is a crucial issue in deep learning model security research. By understanding the mechanisms behind this transferability, researchers can design more effective adversarial example methods and assess model vulnerabilities. This also provides a theoretical basis for developing more robust defense strategies. Summary of the Invention

[0006] In response to the lack of analytical methods for existing model adversarial vulnerability assessment in data-free black box scenarios, the present invention proposes a method and system for adversarial migration risk assessment. The present invention proposes a method for generating adversarial samples through adversarial weight tuning (AWT), which optimizes adversarial samples and alternative models through adaptive iterative optimization, thereby improving the migration of adversarial samples without the need for additional data. Furthermore, the generated adversarial samples can be used to evaluate the model's ability to defend against adversarial attacks. Finally, a modular design of the above method can be used to obtain an adversarial migration risk assessment system. The technical solutions proposed by the present invention are as follows:

[0007] In a first aspect, the present invention proposes a method for assessing risk of migration based on a model with weight adjustment, comprising:

[0008] 1) Obtaining a target model and its input sample set, and initializing a first surrogate model of the target model; the target model is used in the field of image recognition, the input sample is an image, and the first surrogate model and the target model achieve the same function;

[0009] 2) Obtain the current adversarial example and the current first alternative model to be optimized in this iteration; if this is the first iteration, the current adversarial example is the input sample in the input sample set, and the current first alternative model is the initialized first alternative model; if this is not the first iteration, the current adversarial example and the current alternative model are the updated adversarial example and the first alternative model in the previous iteration, respectively;

[0010] 3) Optimize and update the current adversarial sample based on the current first alternative model;

[0011] 4) Optimize and update the weight parameters of the current first alternative model based on the updated adversarial sample;

[0012] 5) Repeat steps 2)-4) to iteratively optimize the adversarial sample and the first alternative model until the preset iteration rounds are completed to obtain the final adversarial sample;

[0013] 6) Repeat steps 2)-5) to traverse the input sample set to obtain multiple final adversarial samples to form the adversarial sample set;

[0014] 7) Reinitialize the second alternative model of the target model, input the obtained adversarial sample set into the target model and the re-acquired second alternative model, calculate the security index of the target model based on the outputs of the target model and the second alternative model, the lower the security index, the more vulnerable the target model is to transfer adversarial attacks, evaluate the adversarial transfer risk of the target model based on the security index, and output the evaluation result.

[0015] In a second aspect, the present invention also proposes a model-based weight adjustment-based anti-migration risk assessment system for implementing the above method.

[0016] Compared with the prior art, the present invention has the following beneficial effects:

[0017] (4) In a more realistic and fragile black-box scenario, i.e., when the attacker cannot directly access the target model, the vulnerability of the model to adversarial sample attacks is analyzed;

[0018] (5) This invention provides an innovative method for generating adversarial examples with strong transferability without relying on additional datasets. This method saves the time and resources required to obtain and process additional data, and improves the efficiency and practicality of generating adversarial examples, making security testing of models against adversarial example attacks more convenient and economical.

[0019] (6) This paper proposes a security metric for measuring a model’s ability to resist generative adversarial attacks, providing a quantitative measure of the model’s security. This metric allows researchers and engineers to evaluate the security of different models or the effectiveness of different defense strategies. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 This is the architecture diagram of the model-based adversarial migration risk analysis system based on adversarial weight adjustment;

[0021] Figure 2 Flowchart of the model-based adversarial migration risk analysis method based on adversarial weight adjustment;

[0022] Figure 3 Flowchart of the adversarial sample generation method based on adversarial weight adjustment;

[0023] Figure 4 This is a flow chart of the model safety assessment method based on the degree of disturbance of the model prediction value;

[0024] Figure 5 This is a relationship diagram between the adversarial sample migration index and the security index evaluated by the present invention. DETAILED DESCRIPTION

[0025] The present invention will be further described in detail below with reference to the accompanying drawings and examples. It should be noted that the following examples are intended to facilitate understanding of the present invention and do not have any limiting effect on the present invention.

[0026] In one embodiment provided by the present invention, Figure 1 The diagram shows the architecture of the model adversarial migration risk analysis system of the present invention, which is mainly divided into two modules: the adversarial sample module based on adversarial weight adjustment and the model security assessment module based on the degree of disturbance of the model prediction value. Figure 2 The following is a flowchart of the analysis method corresponding to the system. The following is an introduction to the analysis system through two modules:

[0027] 1. Adversarial Sample Module Based on Adversarial Weight Adjustment

[0028] The main purpose of this module is to generate adversarial samples for evaluating the model's ability to defend against generative adversarial attacks, so as to evaluate the model's security. The main process is as follows: Figure 3 As shown. In a black-box scenario where the target model parameters are unknown, generating samples with flat local maxima or using a surrogate model with parameter smoothness can enhance the generalization ability of adversarial samples between different models. The method of the present invention combines the advantages of previous input gradient-based and surrogate model-based methods. Through the dual optimization of input samples and surrogate models, it realizes the transfer adversarial sample attack of the target model in a black-box scenario without auxiliary data. An optional implementation method includes the following steps:

[0029] (1) Obtaining a target model and multiple input samples thereof, wherein the multiple input samples constitute a clean data set, and constructing a first surrogate model having the same function as the target model, and performing the following iterative optimization based on the input samples and the first surrogate model; in this embodiment, the target model is an image classification model, and a commonly used convolutional neural network structure is selected to construct the image classification model as the first surrogate model;

[0030] (2) For a certain input sample x, randomly sample N times in the ζ neighborhood of x to obtain N samples The ζ neighborhood is a hypersphere with the input sample x as the center and ζ as the radius in the sample parameter space. In this embodiment, ζ=3∈, and ∈=16.0 / 255;

[0031] (3) Calculate each random sample The first-order gradient in To find the derivative symbol, For the current first alternative model in the sample The cross entropy loss on , y is the sample label, are the parameters of the current first alternative model;

[0032] (4) Order Estimation Sample The second-order gradient in is the current first alternative model in sample x′ i Cross entropy loss on ;

[0033] Update adversarial example gradients in, is the adversarial sample gradient after the current iteration and the previous iteration update, ω is the balancing factor for gradient smoothing, and N is the number of random samples. In this example, ω = 0.5, N = 20,

[0034] (5) After repeated sampling N times, update the adversarial sample gradient of this round In this embodiment, the gradient attenuation coefficient μ = 1.0, g0 = 0;

[0035] (6) Update adversarial samples is the ε neighborhood of the sample parameter space (a hypersphere with the input sample x as the center and ε as the radius), represents continuous multiplication, x′ t is the adversarial sample after the previous iteration, sign(.) is the sign function, if g t+1 is a positive number, then sign(g t+1 )=1, otherwise sign(g t+1)=-1, α is the step size of adversarial sample gradient update. t+1 Projected into the parameter space of ∈ constraint, ||xx′|| p ≤∈. In this embodiment, α=1.6, p=∞, that is, the infinite norm is used to constrain the absolute maximum value of the elements in the vector;

[0036] (7) Calculate the loss function of the first alternative model Computing model parameter perturbations And update the first alternative model parameters in, is the cross entropy loss of the current first alternative model on the input sample x, is the current first replacement model in the adversarial sample x′ t+1 The approximate value of the cross entropy loss on , the approximate value calculation formula is In this embodiment, the gradient scaling factor β=0.005, and the learning rate lr=0.002.

[0037] (8) Repeat steps (2)-(7) for T rounds to obtain the final generated adversarial sample, defined as In this embodiment, T=10.

[0038] (9) Repeat steps (2)-(8) for the input samples in the clean dataset to obtain the adversarial sample set D. For each input sample, the current first alternative model to be optimized in the corresponding first round of iteration is the first alternative model initialized in step (1).

[0039] 2. Model security assessment module based on the degree of disturbance of model prediction values

[0040] Using the adversarial sample set D generated by module 1, we can further conduct model adversarial attack risk assessment. The main process is as follows: Figure 4 By sampling model parameters near the target model, simulating the alternative model, and inputting the generated adversarial attack samples into each model, the adversarial attack security index is calculated. Evaluate model security. Specifically, an optional implementation includes the following steps:

[0041] (3) Taking the parameters of the target model as θ t As the center, with ε as the radius, randomly sample k times in the parameter space to obtain the second alternative model parameters Here ε∈{0.001, 0.01, 0.1}, k=20;

[0042] (4) Input the adversarial samples in the adversarial sample set into the target model and k second alternative models obtained by random sampling, and calculate the security index of the target model based on the output of the target model and the second alternative model

[0043]

[0044] in, represents the expected value of the probability distribution. Indicates The output of the second alternative model is the weight parameter, Represents the output of the target model.

[0045] In this embodiment, the expected value of the probability distribution is calculated by averaging and approximating using the following method:

[0046] (2.1) For each adversarial example calculate in Indicates that at all sampling values ​​η i The sum is calculated under each sampling value η i Corresponding to a second alternative model, Indicates The output of the second alternative model is the weight parameter, represents the output of the target model, ||.||2 represents the L2 norm, It means element-by-element addition;

[0047] (2.2) Calculating the model's security index against attacks

[0048] To verify the effectiveness of the present invention, the above measures were implemented on the ImageNet dataset, and experiments were conducted on models based on convolutional neural networks (CNNs), such as ResNet-50, ResNet-101, and Inception-v3, as well as models based on transformers, such as Swin-T, Swin-S, and LeViT. The attack success rate (ASR) was selected to evaluate the performance of the adversarial sample generation method proposed in this invention, and the Pearson and Spearman correlation coefficients were selected to evaluate the security index of the adversarial attack. rationality.

[0049] (4) Based on safety index Evaluate the adversarial transfer risk of the target model, including four levels: security Relatively safe There are risks There are serious risks

[0050] First, the results in Table 1 show the attack success rate of the adversarial samples on the target model, using the convolutional neural network-based model as the replacement model and all six models as the target model. 20 adversarial sample images were generated on the replacement model respectively. In order to reduce the experimental error, the experiment was repeated 10 times, and the mean and standard deviation of the attack success rate are given in the table. According to the results in Table 1, the adversarial sample generation method proposed in the present invention not only has strong transferability between models with similar network structures, but also can achieve a good attack success rate in scenarios with heterogeneous network structures. This shows that the neural network model still has the risk of migratable adversarial sample attacks in the black box scenario, and in previous experiments, the security of the model has been overestimated.

[0051] Table 1

[0052]

[0053] Secondly, in order to prove the model's security index against attacks To verify the rationality of the proposed method, we used ResNet-50 as the replacement model and VGG16 as the target model, and generated 100 adversarial samples. Figure 5 As shown, the y-axis is the migration index of the adversarial sample, and the calculation method is in is the confidence of the original sample in the correct class output by the model, The confidence level of the adversarial sample in the correct class output by the model measures the decrease in the expected value of the model classification on the target class. The x-axis is the security index calculated by the method of the present invention. By calculating the Pearson correlation coefficient between the two, it is -0.64 and the Spearman correlation coefficient is -0.68. There is a negative correlation between the two, which means that the higher the success rate of the model under adversarial sample attack, the lower the security index of the model. The lower it is, the better the method proposed by the present invention is.

[0054] Third, in order to evaluate the security of the model's ability to resist generative adversarial attacks, ResNet-50 and ResNet-101 are used as target models. Under the assumption that the perturbation range ε∈{0.001, 0.01, 0.1} is allowed, the model's adversarial attack security index is calculated. Table 2 shows the security index obtained based on different adversarial sample generation methods. According to the results in the table, in all scenarios, when the adversarial weight adjustment method proposed in the present invention is used to generate adversarial samples, the security index of the model is the lowest. This also shows that the method and system in the present invention can better evaluate the vulnerabilities of the model.

[0055] Table 2

[0056]

[0057] The above-described embodiments merely illustrate several implementations of the present invention, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present invention. Persons skilled in the art will readily appreciate that variations and modifications may be made without departing from the scope of the present invention, all of which fall within the scope of protection of the present invention.

Claims

1. A model-based weight adjustment-based risk assessment method for migratory resistance, characterized in that: include: 1) Obtain the target model and its input sample set, and initialize the first alternative model of the target model; The target model is used in the field of image recognition, the input sample is an image, and the first replacement model and the target model realize the same function; 2) Obtain the current adversarial sample and the current first alternative model to be optimized in this round of iteration; if this round is the first round of iteration, the current adversarial sample is the input sample in the input sample set, and the current first alternative model is the initialized first alternative model; if this round is not the first round of iteration, the current adversarial sample and the current alternative model are respectively the adversarial sample and the first alternative model updated in the previous round of iteration; 3) Optimize and update the current adversarial sample based on the current first alternative model; 4) Optimize and update the weight parameters of the current first alternative model based on the updated adversarial sample; 5) Repeat steps 2)-4) to iteratively optimize the adversarial sample and the first alternative model until the preset iteration rounds are completed to obtain the final adversarial sample; 6) Repeat steps 2)-5) to traverse the input sample set to obtain multiple final adversarial samples to form an adversarial sample set; 7) Reinitialize the second alternative model of the target model, input the obtained adversarial sample set into the target model and the re-acquired second alternative model, calculate the security index of the target model according to the outputs of the target model and the second alternative model, the lower the security index, the more vulnerable the target model is to transfer adversarial attacks, evaluate the adversarial transfer risk of the target model based on the security index, and output the evaluation result.

2. The method for assessing risk of model resistance to migration based on weight adjustment according to claim 1, characterized in that: The optimization and update of the current adversarial sample based on the current first alternative model is specifically as follows: 3.1) Randomly sample N samples in the ζ neighborhood of the current adversarial sample The ζ neighborhood is a hypersphere with the current adversarial sample as the center and ζ as the radius in the sample parameter space; 3.2) Calculate the average adversarial sample gradient of this round based on the N samples obtained by random sampling. The calculation formula is: in, is the adversarial sample gradient after the update of this iteration and the previous iteration, t≥0, ω is the balancing factor for gradient smoothing, g i ′ For sample The first-order gradient of is the gradient operator, For the current first alternative model in the sample The cross entropy loss on , y is the sample label, is the weight parameter of the current first alternative model, For sample The approximate second-order gradient of For the current first alternative model in sample x i ′ The cross entropy loss on , α is the step size of adversarial sample gradient update, ||.||1 is the L1 norm; 3.3) Calculate the final adversarial sample gradient based on the average adversarial sample gradient. The calculation formula is: where g t is the final adversarial sample gradient after the previous round of iterative update, g0=0, μ is the gradient attenuation coefficient; 2.4) Update adversarial examples is the ∈ neighborhood of the sample parameter space, 3∈=ζ, Indicates continuous multiplication, x t ′ is the adversarial sample after the previous iteration update, and sign(.) is the sign function.

3. The method for assessing risk of model resistance to migration based on weight adjustment according to claim 1, characterized in that: The optimization and updating of the weight parameters of the current first alternative model based on the updated adversarial sample is specifically as follows: 4.1) Calculate the overall loss of the current first alternative model The calculation formula is: in is the current first alternative model in the adversarial sample x t ′ +1 The loss on is the cross entropy loss of the current first alternative model on the input sample x; 4.2) Calculate the model parameter perturbation of the current first alternative model The calculation formula is: in is the weight parameter of the current first alternative model, β is the gradient scaling factor; 4.3) Update the weight parameters of the first alternative model, and obtain the first alternative model after this round of update. The calculation formula for updating the weight parameters is: Among them, lr is the learning rate.

4. The method for assessing risk of model resistance to migration based on weight adjustment according to claim 1, characterized in that: In step 7), the second alternative model for reinitializing the target model is specifically: Get the weight parameter θ of the target model t , defined in parameter space with θ t A hypersphere Ω with ε as the radius is centered, and the parameter η is obtained by random sampling multiple times in the hypersphere Ω. i ∈Ω, θ t With η i Adding them element by element, we get That is the weight parameter of the second alternative model, combined with the weight parameter and a model architecture of the target model to obtain second substitute models of multiple target models.

5. The method for assessing risk of model resistance to migration based on weight adjustment according to claim 4, characterized in that: In step 7), the safety index of the target model is calculated according to the output of the target model and the second alternative model, and the calculation formula is: in, is the transferability adversarial attack security index of the target model, represents the expected value of the probability distribution, and the calculation range of the probability distribution is all adversarial samples in the adversarial sample set and all second alternative models obtained by reinitialization. ||.||2 represents the L2 norm. is the adversarial sample obtained by iterative optimization, D is the adversarial sample set, Indicates The output of the second alternative model is the weight parameter, Represents the output of the target model.

6. The method for assessing risk of model resistance to migration based on weight adjustment according to claim 5, characterized in that: In step 7), the adversarial migration risk of the target model is evaluated based on the security index, specifically by using a preset threshold to divide the adversarial migration risk of the target model into multiple levels.

7. A model migration risk assessment system based on adversarial weight adjustment, characterized in that: For implementing the method according to claim 1, the system comprises: An adversarial sample generation module is used to obtain the input sample of the target model and the first alternative model, iteratively optimize them, and output an optimized adversarial sample set; The model security assessment module is used to obtain the second alternative model, calculate and output the security index of the target model based on the target model, the second alternative model and the adversarial sample set, and is also used to evaluate the adversarial migration risk of the target model based on the security index and output the assessment result.

Images