Black box model verification method based on controllable migration adversarial sample

Abstract

The invention provides a black box model verification method based on a controllable migration adversarial sample, and the method comprises the steps: selecting a source model of a white box, and obtaining a group of white box shielding models of each candidate model type according to the candidate model types; generating an adversarial sample of a candidate category based on the source model and a group of shielding models of the candidate category, and optimizing adversarial disturbance by utilizing gradient information of the source model and the shielding models in a complementary mode in the generation process; respectively inputting the generated adversarial samples of the candidate model categories into the black box model to be verified, testing the output of the black box model, selecting the model category corresponding to the adversarial sample with the minimum output performance reduction as a verification model category, and if the verification model category is consistent with the target model category, judging whether the model category is consistent with the target model category. And if yes, the model service provider provides correct model service. According to the invention, an efficient and reliable solution is provided for model authenticity authentication in an MLaaS scene.

Description

Technical Field

[0001] The present invention relates to information security technology, and in particular to a black box classification model verification technology. Technical Background

[0002] In recent years, with the rapid development of artificial intelligence technology, the Machine Learning as a Service (MLaaS) model has become mainstream. Technology companies such as Google, Amazon, and Microsoft provide users with prediction services based on pre-trained models through cloud-based APIs. Users can easily call high-performance models to solve practical problems without having to bear the complexity and high cost of model training. However, the openness of MLaaS also brings new challenges: it is difficult for users to verify whether the models provided by service providers are consistent with their claims. Because the internal structure and parameters of the model are completely black-boxed, users lack effective means to verify the authenticity of the model. This not only harms user rights and interests, but may also pose potential risks to key business scenarios that rely on model output (such as financial risk assessment and medical diagnosis).

[0003] In the field of model verification, existing methods often rely on the statistical properties of model outputs or the analysis of specific input-output patterns, but these methods have significant limitations. For example, verification based on confidence distributions is susceptible to model calibration bias, while solutions based on functional testing have difficulty distinguishing between models with similar architectures but significant performance differences. Summary of the Invention

[0004] The technical problem to be solved by the present invention is to propose a black box verification solution that can distinguish different model types.

[0005] The technical solution adopted by the present invention to solve the above technical problems is a black box model verification method based on controllable transfer adversarial samples, comprising the following steps:

[0006] Receive the target model category of the black box model to be verified; select a white box source model, and then obtain a set of white box shielding models for each candidate model category based on the candidate model category;

[0007] Based on the source model and a set of shielded models for a candidate class, adversarial examples for the candidate class are generated. During the generation process, the gradient information of the source model and the shielded model are used in a complementary manner to optimize the adversarial perturbation. Specifically, the gradient information of the source model is used to maximize the adversarial strength, and the gradient information of the shielded model is used to reduce the vulnerability of the shielded model itself. This makes the generated adversarial examples offensive to the source model while maintaining the performance of the shielded model.

[0008] The generated adversarial samples of each candidate model category are input into the black-box model to be verified respectively, and the black-box model output is tested. The model category corresponding to the adversarial sample with the smallest output performance degradation is selected as the verification model category. If the verification model category is consistent with the target model category, it means that the model service provider provides the correct model service. Otherwise, it is judged that the model service provider does not provide the correct model service.

[0009] This paper constructs a verification protocol that can distinguish between different model types by generating adversarial examples with model-specific responses and combining them with multi-dimensional attack effect analysis. The core of this method is to leverage the directional failure characteristics of controllable migration adversarial examples as a unique identifier for the model, acting like a fingerprint. By analyzing the black-box model's immunity to specific adversarial examples, its true architecture or version can be inferred.

[0010] The beneficial effect of the present invention is that it breaks through the limitations of traditional verification technology and provides an efficient and reliable solution for model authenticity authentication in MLaaS scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] Figure 1 Validate scenarios for black-box models;

[0012] Figure 2 Flowchart for generating controllable transfer adversarial examples. DETAILED DESCRIPTION

[0013] The tasks of the present invention can be summarized as follows:

[0014] Assume that the black box model to be verified is M target , the service provider claims that it belongs to the target model category M, and the verification goal is to generate an adversarial sample x based on the original sample x adv , according to M target Face x adv Reaction verification M target Whether it truly belongs to M.

[0015] In a common adversarial attack scenario with transferability, let f(x) denote the output of the white-box source model and g(x) denote the output of the black-box target model. By generating an adversarial perturbation δ based on f(x), the adversarial sample x adv = x + δ to mislead both f(x) and g(x). This task is described as an optimization problem: within a given adversarial perturbation budget ε, the adversarial perturbation δ that maximizes the loss function J(f(x+δ)), y) between the output f(x+δ) and the label y of x is defined as the optimal adversarial perturbation δ * :

[0016]

[0017] Among them, |||| is the L1 norm, st represents the constraint condition, and the loss function J can use cross entropy.

[0018] In this scenario, adversarial samples generated based on the white-box source model have universal transferability and are not only effective against the black-box target model, but also against other black-box models. Figure 1 shown.

[0019] In this paper, we propose the concept of controllable transferability of adversarial examples, that is, the transferability of adversarial examples can be controlled. This paper proposes a framework for generating adversarial examples with controllable transferability. The process can be summarized as follows:

[0020] Given a source model M source A set of shielding models corresponding to each candidate model type All are white box settings, K is the total number of sub-models in a set of shielded models. The candidate model type is the sum of different model types that can be distinguished by the present invention. Source model M source There is no strong correlation with the target model category M, so we only need to select a source model M with strong migration attack capability. source , so that the adversarial samples generated based on this model can achieve better attack effects on the target model M. In addition, it is necessary to ensure that source and various types of M Shieldeds When generating transferable adversarial examples, M source Stay consistent.

[0021] First. The next step is to use gradient adversarial technology to balance the preservation of transferability and the reduction of adversarial effectiveness on the shielded model. In this method, the gradient adversarial technology is used from M source to optimize the adversarial perturbation to maximize the attack efficiency while utilizing the gradient information from M Shieldeds gradient information to reduce its vulnerability.

[0022] Based on the above method, the transferability of the generated adversarial samples can be controlled so that they are aggressive to other models without affecting the shielding model M. Shieldeds Generate corresponding adversarial samples for each candidate model type. For a specific candidate model type, determine a set of shielding models based on the type. Each sub-model in the shielding model group conforms to the network structure of the specific candidate model type. Wait for all candidate model types to have controllable transfer test adversarial samples x adv After the generation is completed, each x adv Input the black box model M to be verified target , test M target Each output of Mtarget The smallest performance degradation x adv The corresponding model type is used as the verification model type.

[0023] The flow chart of controllable transfer adversarial sample generation is as follows Figure 2 shown.

[0024] Next, we introduce the details of the two stages of controllable transfer adversarial example generation:

[0025] 1) Generate transferable adversarial samples: In order to source To generate transferable adversarial samples, this paper proposes a novel adversarial sample generation method, which uses the Pay No Attention (PNA) method.

[0026] The present invention designs the attention-free mechanism PNA based on ViTs (vision transformer) and enhances the transferability of adversarial samples by reducing the interaction between attention heads in ViTs during back-propagation.

[0027] The existing ViTs (Vision Transformer) network architecture is based on the self-attention mechanism and the multi-layer perceptron (MLP), which are the core components of ViTs. The self-attention mechanism consists of a self-attention layer and a query-key-value (QKV) layer. The self-attention mechanism is used to capture features after attention, and the MLP performs nonlinear transformations on these features to enhance the model's expressiveness.

[0028] After the adversarial samples are generated, they are effective against both CNN and ViT, and can successfully attack contemporary mainstream models. In the existing self-attention mechanism, the output X ′ It is usually calculated as:

[0029]

[0030] Where A represents the attention mechanism, W Q 、W K and W V are the query, key, and value weight matrices, respectively. The key modification of PNA is to ignore the gradient of the attention mechanism A, simplifying the backpropagation process and reducing the reliance on model-specific attention structures. The PNA gradient is approximately expressed as:

[0031]

[0032] in represents the identity matrix. This method effectively ignores the gradient from the attention mechanism and focuses on the value projection WV , thereby improving the transferability against perturbations.

[0033] The core of PNA is to ignore the gradient of the self-attention layer in the ViT model, which is equivalent to dividing the gradient of the VIT model into three parts. PNA only takes the remaining two parts, the QKV layer and the MLP layer. In other words, PNA is a selective processing of the gradient.

[0034] This paper further improves the transferability of adversarial examples by modifying the gradient flow in the key layer. Inspired by the Skip Gradient Method (SGM), this paper introduces gradient modulation in the MLP layer and the QKV layer. Specifically, a gradient is generated that is consistent with the input gradient of the MLP layer. Mask M of the same shape MLP , where each element is assigned a preset decay factor γ, γ∈(0, 1). The output gradient of the modified MLP layer is given by:

[0035]

[0036] This method reduces the model’s reliance on specific feature representations in the MLP layer, thereby enhancing the generalization of adversarial perturbations. Similarly, for the QKV layer, a gradient with the same shape as the input of the QKV layer is generated. The same mask M QKV , And applied to the corresponding gradient processing. By refining the gradient flow in these key layers, the generated adversarial examples show better robustness in different model architectures, including ViTs and CNNs.

[0037] 2) Gradient adversarial:

[0038] To achieve controllable transferability, we introduce Gradient Antagonism, which aims to ensure that adversarial examples preserve their transferability while minimizing their impact on the shielded model.

[0039] Given a set of shielding models We refine previously generated adversarial examples to reduce their effectiveness on the shielded model while maintaining attack performance on other targets. This approach ensures that the shielded model remains robust under adversarial attacks, thus achieving controllable and transferable performance.

[0040] The core idea of ​​gradient adversarial is to optimize the adversarial perturbation by utilizing the gradient information of the source model and the shielded model in a complementary way. Specifically, we use the gradient of the source model To enhance the strength of the adversarial model, while using the reverse gradient of the shielding model To reduce the adversarial effects of these models. The final adversarial perturbation δ * It is obtained by solving the following optimization problem:

[0041]

[0042] st||δ||≤∈

[0043] where λ is a balancing coefficient that controls the trade-off between maximizing the attack strength on the source model and suppressing the adversarial influence on the shielding model. k Represented as each shielding model The weights assigned allow different levels of importance to be assigned to each shielded model. This method ensures that the adversarial perturbation remains effective on the target model while reducing its transferability to the shielded model, thereby achieving controlled adversarial transfer. adv =x+δ * is the final adversarial sample.

[0044] Specifically, the workflow of the model for generating adversarial samples is as follows:

[0045] Input: Based on the optimization problem, a loss function J is established, and the loss function J is a white box source model M. source , a set of shielding models M Shieldeds , a clean image x as the original sample and its label y.

[0046] Parameters: adversarial perturbation budget ε, number of iterations I, balance coefficient λ.

[0047] Output: Adversarial example x adv .

[0048] Preliminary preparation:

[0049] a) δ0←0, initialize the adversarial perturbation δ0 to 0, A←B means using B to assign A;

[0050] b) Set the adversarial perturbation step size α to be the adversarial perturbation budget ∈ divided by the number of iterations I;

[0051] Iterative steps, for the i-th iteration, perform the following operations:

[0052] c) In the MLP layer, calculate the gradient of the MLP layer

[0053] d) In the QKV layer, calculate the gradient of the QKV layer

[0054] e)g source ←PNA(Msource (x+δ i ),(x+δ i )), to maximize J(M source (x+δ i ),y) is the target, the output of PNA(M source (x+δ i ),(x+δ i )) to update, use the updated PNA to output PNA(M source (x+δ i ),(x+δ i )) as the gradient g source ;

[0055] f) According to the weight μ of each model k Calculate the protected model M Shieldeds The gradient g Shieldeds , Indicates the i Calculate the gradient, for example, the current M Shieldeds Designated as resnet category, through μ k You can adjust various models in the resnet category

[0056] For example, you can strengthen the protection of the resnet18 model.

[0057] The gradient of the resnet18 model uses a larger μ k , other models under the resnet category choose smaller μ k ;g)g←g source -λg Shieldeds , the source model gradient g source Subtract the weighted shielded model gradient

[0058] λg Shieldeds , calculate the comprehensive gradient g;

[0059] h)δ i+1 ←clip ∈ (δ i +α·g), using ∈-clipping to exploit local iterative adversarial perturbation δ i The sum of the multiplication results of the step size and the comprehensive gradient g is used to update the adversarial perturbation δ for the next iteration i+1 ; clip is a clipping function, which is a function that limits the value range element by element. It is often used to constrain the input value to a specified interval. ∈ Indicates that ∈ is the maximum value constraint;

[0060] i) Determine whether the number of iterations reaches i+1 or equals I. If so, the iteration ends. Otherwise,

[0061] Update i=i+1 and return to iterative step c);

[0062] After the iteration, we finally get:

[0063] j)x adv ←x+δ I , calculate the final adversarial sample x adv .

[0064] The x of each candidate model category adv Input the black box model M to be verified target , test M target Output. Select M target The performance degradation is the smallest, indicating that M target Subject to x adv The effect is minimal, which proves that M target Belong to the model category, and use the category as the verification model category. Determine whether the verification model category is the same as the target model category M. If so, the model service provider provides the correct model service. Otherwise, the model service provider does not provide the correct model service as required.

Claims

1. A black-box model verification method based on controllable transfer adversarial samples, characterized by: Including steps: Receive the target model category of the black box model to be verified; select a white box source model, and then obtain a set of white box shielding models for each candidate model category based on the candidate model category; Based on the source model and a set of shielded models for a candidate class, adversarial examples for the candidate class are generated. During the generation process, the source model and the shielded model are used in a complementary manner to optimize the adversarial perturbation. Specifically, the gradient information of the source model is used to maximize the adversarial strength, and the gradient information of the shielded model is used to reduce the vulnerability of the shielded model itself. This makes the generated adversarial examples offensive to the source model while maintaining the performance of the shielded model. The generated adversarial samples of each candidate model category are input into the black-box model to be verified respectively, and the black-box model output is tested. The model category corresponding to the adversarial sample with the smallest output performance degradation is selected as the verification model category. If the verification model category is consistent with the target model category, it means that the model service provider provides the correct model service. Otherwise, it is judged that the model service provider does not provide the correct model service.

2. The method according to claim 1, wherein: The specific optimization problem of optimizing the adversarial perturbation by utilizing the source model and the shielding model in a complementary manner is expressed as: st||δ||≤∈ Among them, δ * is the optimal adversarial perturbation, δ is the adversarial perturbation variable, J is the loss function, x is the input sample, y is the label of the input sample, M source is the source model, is the kth shielding model in a set of shielding models, K is the total number of models in a set of shielding models, λ is the balance coefficient, μ k Represented as the kth shielding model The weight of ;|||| is the L1 norm, st represents the constraint condition, ∈ is the given adversarial perturbation budget ε; The final adversarial sample x adv =x+δ * .

3. The method according to claim 2, wherein: The source model uses the ViTs network model without attention mechanism; The source model uses the PNA gradient calculation method as follows: X′ is the output of the source model, X is the input of the source model, A is the attention mechanism, W V is the value weight matrix, T is the transpose, is the identity matrix.

4. The method according to claim 3, wherein: In the ViTs network model without attention mechanism, gradient modulation is introduced in the multi-layer perceptron MLP layer and self-attention QKV layer; Generate a gradient with the input of the MLP layer Mask M of the same shape MLP Correction is performed, and the MLP layer outputs the gradient after correction for: Generate a gradient with the input of the QKV layer Mask M of the same shape QKV After correction, the QKV layer outputs the gradient for:

5. The method according to claim 4, wherein: Based on the source model and a set of shielded models of a candidate class, a model of an adversarial sample of the candidate class is generated. The following steps are performed: Receive input loss function J, white box source model M source , a set of shielding models M Shieldeds , a set of K shielding models The original x and label y; set the adversarial perturbation budget ε, the total number of iterations I and the balance coefficient λ; Initialization step: Initialize the current iteration number i = 1, initialize the adversarial perturbation δ i 0; set the anti-perturbation step size Iterative steps, for the i-th iteration, perform the following operations: In the MLP layer, calculate the output gradient of the modified MLP layer In the QKV layer, calculate the corrected QKV layer output gradient Calculate the source model M using the PNA method source The output gradient g source :To maximize J(M source (x+δ i ),y) is the target, the output of PNA(M source (x+δ i ),(x+δ i )) to update, use the updated PNA to output PNA(M source (x+δ i ),(x+δ i )) as the gradient g source ; Calculate shielding model M Shieldeds The output gradient g Shieldeds : Calculate the comprehensive gradient g as the source model gradient minus the weighted shielding model gradient, g = g source -λg Shieldeds ; Using local iteration to counter the perturbation δ i The sum of the multiplication results of the step size and the comprehensive gradient g is used to update the adversarial perturbation δ of the next iteration using ∈-clipping i+1 , δ i+1 =clip ∈ (δ i +α·g); clip is a clipping function, which is a function that limits the value range element by element. ∈ Indicates that ∈ is the maximum value constraint; Determine whether the number of iterations reaches i+1 and is equal to I. If so, the iteration ends. Otherwise, update i=i+1 and return to the iteration step. After the iteration, we finally get: x adv =x+δ I , δ I As the best adversarial perturbation, the final adversarial sample x is calculated adv .

Images