A black-box adversarial robustness evaluation method based on an image classification model

Abstract

The present application belongs to the field of intelligent security, and particularly relates to a black-box adversarial robustness evaluation method based on an image classification model; comprising: using an image classification model to classify and process images of different categories to obtain a plurality of image sample sets of different categories; processing the plurality of image sample sets of different categories to obtain a decision boundary image sample set; converting the decision boundary image sample set into a decision boundary image sample matrix; singular value decomposing the decision boundary image sample matrix to obtain singular values of the decision boundary image sample matrix; calculating the decision boundary complexity according to the singular values; normalizing the decision boundary complexity to obtain a robustness evaluation index; evaluating the robustness of the image classification model according to the robustness evaluation index to obtain a robustness evaluation result of the image classification model; the present application does not require the structure and parameter information of the target model, and greatly improves the security of the adversarial robustness evaluation process of the image classification model.

Description

Technical Field

[0001] The present invention belongs to the field of intelligent security, and in particular relates to a black box adversarial robustness evaluation method based on an image classification model. Background Art

[0002] In recent years, deep neural networks (DNNs) have made tremendous progress in a range of machine learning tasks, including computer vision, speech recognition, and natural language processing. While deep learning has achieved remarkable results in many important areas, security issues remain. Specifically, DNNs are susceptible to perturbations imperceptible to the human eye, which can mislead image classification models into outputting incorrect prediction labels. These perturbed examples, misclassified by DNNs, are called adversarial examples. The existence of adversarial examples poses significant security risks in sensitive tasks with high security requirements.

[0003] As research on adversarial examples deepens, researchers have discovered that they are ubiquitous in nearly all deep learning models, leading to a significant increase in the research on adversarial example generation algorithms. Furthermore, defending against adversarial attacks and reducing the likelihood of image classification models outputting incorrect labels for adversarial examples have also garnered significant attention. Researchers have proposed a range of defensive measures, including gradient masking, adversarial training, data processing, and feature compression. While these defensive measures can effectively defend against certain adversarial attacks, comprehensive adversarial robustness assessment methods are currently lacking to scientifically and effectively evaluate these defenses.

[0004] Current adversarial robustness assessment primarily involves continuously testing models against attack and defense using advanced adversarial attack algorithms, measuring model robustness through basic metrics such as classification accuracy, query count, or attack strength. These methods rely heavily on the sophistication of the adversarial attack, and conventional adversarial attack methods have varying degrees of limitations, making them inapplicable to all deep neural networks. Furthermore, the strength of the attack can significantly influence the assessment process, leading to inaccurate results. This interdependence between adversarial robustness assessment and attack methods can lead to biased analysis.

[0005] At present, there are two main solutions to the above problems: (1) through explanatory research on the existence of adversarial samples, improve the sophistication of adversarial attack methods and eliminate conditional restrictions; (2) separate the adversarial robustness evaluation task from the adversarial attack.

[0006] The decision boundary (DB) is the surface in the model's input space where the output prediction varies between different categories. It serves as the boundary between correct and incorrect classifications in deep neural network classification tasks. Classification models are trained on raw data and determine a decision boundary in the decision space that accurately classifies training examples. Nodes on one side of the decision boundary are assigned to one class, and nodes on the other side are assigned to the other class. Attackers can perturb the decision boundary to create adversarial examples. These perturbations cause the original examples to cross the classification model's decision boundary, leading to incorrect classification. Therefore, the decision boundary directly reflects the model's classification basis. A well-designed decision boundary can improve the generalization ability and robustness of deep neural network models. The connection between a model's decision boundary and adversarial robustness has been largely unexplored, offering a new approach to designing metrics for evaluating model adversarial robustness. Summary of the Invention

[0007] In view of the shortcomings of the existing technology, the present invention proposes a black box adversarial robustness evaluation method based on an image classification model, which includes:

[0008] S1: Use the image classification model to classify images of different categories to obtain multiple image sample sets of different categories;

[0009] S2: Use the decision boundary sample search strategy to process multiple image sample sets of different categories to obtain a decision boundary image sample set;

[0010] S3: Convert the decision boundary image sample set into a decision boundary image sample matrix;

[0011] S4: Perform singular value decomposition on the decision boundary image sample matrix to obtain the singular values ​​of the decision boundary image sample matrix;

[0012] S5: The decision boundary complexity is calculated based on the singular values ​​of the decision boundary image sample matrix;

[0013] S6: Normalize the decision boundary complexity to obtain the robustness evaluation index;

[0014] S7: Perform robustness evaluation on the image classification model according to the robustness evaluation index to obtain the robustness evaluation result of the image classification model.

[0015] Preferably, in step S2, the process of obtaining the decision boundary image sample set includes:

[0016] S21: Randomly take two image samples x from category i and category j of the image dataset i and x j ;

[0017] S22: Get the suspicious decision boundary sample x by averaging the corresponding pixel values ​​of the two image samples db ;

[0018] S23: Determine the suspicious decision boundary sample x db Whether the confidence output of satisfies the confidence condition, if so, execute step S24, otherwise, execute step S25;

[0019] S24: The suspicious decision boundary sample x db Determine it as a decision boundary sample and add it to the decision boundary sample set of class i and class j;

[0020] S25: Determine f i (x db )>f j (x db ) is satisfied, if so, then the sample x i Update to sample x db Otherwise, the sample x j Update to sample x db And return to step S22; wherein, f i (x) represents the confidence output that the image sample x belongs to the i-th category, f j (x) represents the confidence output that the image sample x belongs to the jth category;

[0021] S26: Determine whether the number of samples in the suspicious decision boundary sample set is greater than the set value N. If it is greater than N, obtain the final i-th and j-th decision boundary sample sets; otherwise, repeat steps S21-S26;

[0022] Furthermore, the confidence condition is expressed as:

[0023] f i (x)-f j (x)≤γ,f i (x)=max k∈C f k (x)

[0024] Among them, C represents the total number of categories of the image dataset, f k (x) represents the confidence output that the image sample x belongs to the kth category, and γ represents the threshold.

[0025] Preferably, the process of converting the decision boundary image sample set into a decision boundary image sample matrix includes:

[0026] Expand the pixel values ​​of N decision boundary samples in the decision boundary sample set into one column to obtain N columns of pixel data; expand the pixels of each decision boundary sample set to obtain N columns of pixel data; splicing N columns of pixel data are used to obtain the decision boundary matrix X, where C is the total number of image categories.

[0027] Preferably, the decision boundary matrix is ​​subjected to singular value decomposition and expressed as:

[0028]

[0029] Where U is the left singular vector matrix of the decision boundary matrix X; Σ is the singular value matrix; V is the right singular value matrix of the decision boundary matrix X; X represents the decision boundary matrix, u nn Represents the element in the nth row and nth column of the left singular value matrix, v mm represents the element in the mth row and mth column of the right singular value matrix, {λ1,…,λ r}The singular values ​​of the decision boundary matrix X, r represents the number of singular values.

[0030] Preferably, the formula for calculating the decision boundary complexity is:

[0031]

[0032] Among them, λ i represents the i-th singular value, r represents the number of singular values, P(λ) represents the probability of the occurrence of the singular value λ, and H(Σ) represents the decision boundary complexity.

[0033] Preferably, the decision boundary complexity is normalized to:

[0034]

[0035] Among them, DBSE represents the adversarial robustness evaluation index, H(Σ) represents the decision boundary complexity, and len(λ) represents the number of singular values ​​in the decision boundary matrix.

[0036] Preferably, performing robustness evaluation on the image classification model according to the robustness evaluation index includes: a larger value of the robustness evaluation index indicates weaker adversarial robustness of the image classification model, and a smaller value of the robustness evaluation index indicates stronger adversarial robustness of the image classification model.

[0037] The beneficial effects of the present invention are as follows: in order to solve the problem that adversarial robustness evaluation depends on the adversarial sample generation method, the present invention utilizes the relationship between the decision boundary of the image classification model and the strength of the model's adversarial robustness to realize the evaluation of the adversarial robustness of the image classification model, abandons the method of using adversarial samples to conduct adversarial testing on the model in the previous evaluation process, eliminates the influence of adversarial samples of different qualities on the evaluation results, and ensures the consistency and credibility of the evaluation results; the present invention designs a new robustness evaluation indicator, thereby optimizing the entire model robustness evaluation process, which can improve the robustness of the image classification model; in addition, the present invention is suitable for white-box scenarios, does not require the structure and parameter information of the target model, and greatly improves the security of the image classification model adversarial robustness evaluation process. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Figure 1 This is a structural diagram of the black-box adversarial robustness evaluation method based on the image classification model in the present invention;

[0039] Figure 2 This is a flow chart of the black-box adversarial robustness evaluation method based on the image classification model in the present invention. DETAILED DESCRIPTION

[0040] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0041] This paper proposes a black box adversarial robustness evaluation method based on image classification model, such as Figure 1 、 Figure 2 As shown, the method includes:

[0042] S1: Use the image classification model to classify images of different categories to obtain multiple image sample sets of different categories.

[0043] Obtain images of different categories to be classified, and use image classification models to classify images of different categories to obtain multiple image sample sets of different categories; store samples of the same category in the same file for subsequent use.

[0044] S2: A decision boundary sample search strategy is used to process multiple image sample sets of different categories to obtain a decision boundary image sample set.

[0045] i, j represent the i-th and j-th categories in the dataset respectively and i∈(0,C-1),j∈(1,C); f i(x) represents the confidence output that the image sample x belongs to the i-th category, f j (x) represents the confidence output that the image sample x belongs to the jth category. By looping through all different category combinations in the dataset with different i and j values, the following steps are performed:

[0046] S21: Randomly take two image samples x from category i and category j of the image dataset i and x j ;

[0047] S22: According to two image samples x i and x j The corresponding pixel values ​​are averaged to obtain the suspicious decision boundary sample x db ; Calculate suspicious decision boundary image samples:

[0048] x db =(x i +x j ) / 2

[0049] S23: Determine suspicious sample x db Whether the confidence condition is met, if so, execute step S24, otherwise execute step S26;

[0050] The decision boundary sample between any two different categories means that for the classification model f(x) of C categories, there exists a sample x db Belongs to the data set X, its i-th classification confidence and j-th classification confidence f i (x db ),f j (x db ) is the maximum classification confidence of the model max k∈C f k (x db ) are equal, we can think that the sample x db Located on the decision boundary. Its mathematical expression is as follows:

[0051]

[0052] Considering the search to obtain the suspicious decision boundary image sample x that meets the above formula db The overhead is too large. The strategy designed by this invention sets a threshold γ. For samples with a confidence difference less than the threshold, it is considered to be near the decision boundary, which can simulate the distribution and structural information of the decision boundary. The mathematical expression is as follows:

[0053]

[0054] Therefore, the confidence condition is:

[0055] fi (x db )-f j (x db )≤γ,f i (x db )=max k∈C f k (x db )

[0056] Among them, f i (x db ) represents the suspicious decision boundary sample x db In the target classification model, the confidence output of the i-th category, γ represents the threshold, which is a minimum value, and C represents the total number of all classification categories.

[0057] Judgment sample x db Whether the confidence condition is met, if so, execute step S24, otherwise, execute step S26.

[0058] S24: The suspicious decision boundary sample x db Determine it as a decision boundary sample and add it to the decision boundary sample set of class i and class j;

[0059] S25: Determine f i (x db )>f j (x db ) is satisfied, if so, then the sample x i Update to sample x db Otherwise, the sample x j Update to sample x db And return to step S22;

[0060] S26: Determine whether the number of samples in the suspicious decision boundary sample set is greater than the set value N. If it is greater than N, obtain the final i-th and j-th decision boundary sample sets; otherwise, repeat steps S21-S26;

[0061] S3: Convert the decision boundary image sample set into a decision boundary image sample matrix.

[0062] The process of converting the decision boundary image sample set into the decision boundary image sample matrix includes:

[0063] Expand the pixel values ​​of N decision boundary samples in the decision boundary sample set into one column to obtain N columns of pixel data; expand the pixels of each decision boundary sample set to obtain N columns of pixel data; splicing N columns of pixel data are obtained to obtain the decision boundary matrix X, where C is the total number of image categories;

[0064] S4: Perform singular value decomposition on the decision boundary image sample matrix to obtain the singular values ​​of the decision boundary image sample matrix.

[0065] The singular vectors and singular value matrices of the decision boundary image sample matrix can be obtained through singular value decomposition to determine the main change direction of the decision boundary and the weights of each change direction. The singular value decomposition of the decision boundary image sample matrix is ​​expressed as:

[0066]

[0067] Where U is the left singular vector matrix of the decision boundary matrix X; ∑ is the singular value matrix; V is the right singular value matrix of the decision boundary matrix X; X represents the decision boundary matrix, u nn Represents the element in the nth row and nth column of the left singular value matrix, v mm represents the element in the mth row and mth column of the right singular value matrix; {λ1,…,λ r}The singular values ​​of the decision boundary matrix X, r represents the number of singular values.

[0068] S5: The decision boundary complexity is calculated based on the singular values ​​of the decision boundary image sample matrix.

[0069] The proportion of the singular values ​​of the decision boundary image sample matrix in the total number of singular values ​​is taken as the probability of the occurrence of the singular value. The Shannon entropy is used to quantify the uncertainty of the singular value. If the singular value Shannon entropy is small, it means that the decision boundary has fewer changing directions and the spatial structure is simpler; otherwise, it means that the decision boundary has more changing directions and the spatial structure is more complex.

[0070] The formula for calculating the complexity of the decision boundary is:

[0071]

[0072] Where, represents the i-th singular value, r represents the number of singular values, P(λ) represents the probability of the occurrence of the singular value λ, and H(∑) represents the decision boundary complexity.

[0073] S6: Normalize the decision boundary complexity to obtain the robustness evaluation index. The specific mathematical expression is as follows:

[0074]

[0075] Among them, DBSE represents the adversarial robustness evaluation index, H(∑) represents the decision boundary complexity, and len(λ) represents the number of singular values ​​in the decision boundary matrix.

[0076] S7: Perform robustness evaluation on the image classification model according to the robustness evaluation index to obtain the robustness evaluation result of the image classification model.

[0077] The robustness of the image classification model is evaluated based on the robustness evaluation index. Specifically, the larger the value of the robustness evaluation index, the weaker the adversarial robustness of the image classification model, and the smaller the value of the robustness evaluation index, the stronger the adversarial robustness of the image classification model. When the robustness evaluation index value meets the user's requirements, the corresponding image classification model is used to classify the image, which can obtain more accurate image classification results. The evaluation process of the present invention is applicable to white-box scenarios and does not require the structure and parameter information of the target model, which greatly improves the security of the image classification model's adversarial robustness evaluation process.

[0078] The above embodiments further illustrate the purpose, technical solutions and advantages of the present invention in detail. It should be understood that the above embodiments are only preferred implementation plans of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made to the present invention within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A black-box adversarial robustness evaluation method based on an image classification model, characterized in that: include: S1: Use the image classification model to classify images of different categories to obtain multiple image sample sets of different categories; S2: Use the decision boundary sample search strategy to process multiple image sample sets of different categories to obtain a decision boundary image sample set; The process of obtaining the decision boundary image sample set includes: S21: Randomly take two image samples x from category i and category j of the image dataset i and x j ; S22: Get the suspicious decision boundary sample x by averaging the corresponding pixel values ​​of the two image samples db ; S23: Determine the suspicious decision boundary sample x db The confidence output of satisfies the confidence condition. If so, step S24 is executed. Otherwise, step S25 is executed. The confidence condition is expressed as: f i (x)-f j (x)≤γ,f i (x)=max k∈C f k (x) Among them, C represents the total number of categories of the image dataset, f k (x) represents the confidence output that the image sample x belongs to the kth category, and γ represents the threshold; S24: The suspicious decision boundary sample x db Determine it as a decision boundary sample and add it to the decision boundary sample set of class i and class j; S25: Determine f i (x db )>f j (x db ) is satisfied, if so, then the sample x i Update to sample x db Otherwise, the sample x j Update to sample x db And return to step S22; wherein, f i (x) represents the confidence output that the image sample x belongs to the i-th category, f j (x) represents the confidence output that the image sample x belongs to the jth category; S26: Determine whether the number of samples in the suspicious decision boundary sample set is greater than the set value N. If it is greater than N, obtain the final i-th and j-th decision boundary sample sets; otherwise, repeat steps S21-S26; S3: Converting the decision boundary image sample set into a decision boundary image sample matrix. The process of converting the decision boundary image sample set into the decision boundary image sample matrix includes: Expand the pixel values ​​of N decision boundary samples in the decision boundary sample set into one column to obtain N columns of pixel data; expand the pixels of each decision boundary sample set to obtain N columns of pixel data; splicing N columns of pixel data are obtained to obtain the decision boundary matrix X, where C is the total number of image categories; S4: Perform singular value decomposition on the decision boundary image sample matrix to obtain the singular values ​​of the decision boundary image sample matrix; S5: The decision boundary complexity is calculated based on the singular values ​​of the decision boundary image sample matrix. The formula for calculating the decision boundary complexity is: Among them, λ i represents the i-th singular value, r represents the number of singular values, P(λ) represents the probability of the occurrence of the singular value λ, ∑ is the singular value matrix, and H(∑) represents the decision boundary complexity; S6: Normalize the decision boundary complexity to obtain the robustness evaluation index; the normalized expression for the decision boundary complexity is: Where DBSE represents the adversarial robustness evaluation index, and len(λ) represents the number of singular values ​​in the decision boundary matrix; S7: Performing a robustness evaluation on the image classification model according to the robustness evaluation index to obtain a robustness evaluation result of the image classification model; performing a robustness evaluation on the image classification model according to the robustness evaluation index includes: a larger value of the robustness evaluation index indicates weaker adversarial robustness of the image classification model, and a smaller value of the robustness evaluation index indicates stronger adversarial robustness of the image classification model.

2. The black box adversarial robustness evaluation method based on an image classification model according to claim 1, characterized in that: The singular value decomposition of the decision boundary matrix is ​​expressed as: Where U is the left singular vector matrix of the decision boundary matrix X; ∑ is the singular value matrix; V is the right singular value matrix of the decision boundary matrix X; X represents the decision boundary matrix, u nn Represents the element in the nth row and nth column of the left singular value matrix, v mm represents the element in the mth row and mth column of the right singular value matrix, {λ1,…,λ r }The singular values ​​of the decision boundary matrix X, r represents the number of singular values.

Images